00000100:00100000:0.0:1776188764.862852:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188764.865055:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e01c7a33 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664229184:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776188765.884632:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664229248 00000100:00100000:0.0:1776188765.886391:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664229312 00000100:00100000:0.0:1776188765.886418:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6310 00000100:00100000:0.0:1776188765.886422:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006f76f00e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664229248:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776188765.886436:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776188765.886481:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006f76f00e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664229248:12345-0@lo:13:osp-pre-1-1.0 Request processed in 58us (1902us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188765.886487:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6310 00000100:00100000:0.0:1776188765.886498:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6311 00000100:00100000:0.0:1776188765.886501:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000026c12667 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664229312:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776188765.886522:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776188765.886557:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000026c12667 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664229312:12345-0@lo:13:osp-pre-0-1.0 Request processed in 54us (1893us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188765.886563:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6311 00000100:00100000:0.0:1776188769.659331:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188769.659494:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188769.659530:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188769.659633:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654925888 00000100:00100000:0.0:1776188769.659662:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9472 00000100:00100000:0.0:1776188769.659666:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e310c888 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654925888:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776188769.659692:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e310c888 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654925888:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 26us (163us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188769.659698:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9472 00010000:02020000:0.0:1776188770.381401:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188770.637081:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188770.892422:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188771.010950:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776188771.010966:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000048520f37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664230848:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776188771.011013:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188771.018319:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000048520f37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664230848:0@lo:13:osp-pre-1-1.0 00010000:02020000:0.0:1776188771.149335:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188771.404323:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188771.660404:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188771.916461:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188772.172278:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188772.429284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188772.684285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188772.940675:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188773.196288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188773.452537:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188773.708293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188773.964320:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188774.220298:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188774.481518:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188774.732290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188774.779600:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188774.779642:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188774.779676:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188774.779712:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188774.779742:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654926272 00000100:00100000:0.0:1776188774.779784:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8138 00000100:00100000:0.0:1776188774.779790:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c8b239a6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654926272:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776188774.779855:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c8b239a6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654926272:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 65us (145us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188774.779863:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8138 00010000:02020000:0.0:1776188774.988285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188775.100575:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a635110a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664230976:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776188775.100607:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188775.100677:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664230912 00000100:00100000:0.0:1776188775.100717:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664230976 00000100:00100000:0.0:1776188775.100737:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6316 00000100:00100000:0.0:1776188775.100742:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005e3f10b7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664230912:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776188775.100782:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776188775.103078:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005e3f10b7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664230912:12345-0@lo:13:osp-pre-1-0.0 Request processed in 2326us (2565us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188775.103091:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6316 00010000:02020000:0.0:1776188775.244282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188775.500292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188775.756326:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188776.012295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188776.128165:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664231040 00000100:00100000:0.0:1776188776.128246:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6318 00000100:00100000:0.0:1776188776.128255:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002871fa24 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664231040:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776188776.128280:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776188776.128365:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002871fa24 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664231040:12345-0@lo:13:osp-pre-1-1.0 Request processed in 109us (3729us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188776.128377:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6318 00000100:00100000:0.0:1776188776.128420:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a635110a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664231360:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188776.128464:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188776.128494:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e1f2ed1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664231744:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188776.128523:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188776.128540:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1776188776.128548:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004f9c7e25 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664231104:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188776.132251:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188776.132309:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004d99e80b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664231552:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188776.132342:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188776.133981:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e1f2ed1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664231744:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188776.133998:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a635110a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664231360:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188776.134025:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004f9c7e25 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664231104:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188776.134806:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004d99e80b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664231552:0@lo:400:kworker/u8:0.0 00010000:02020000:0.0:1776188776.270381:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188776.526814:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188776.780335:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188777.036310:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188777.294246:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188777.548316:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188777.804456:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188778.060292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188778.318834:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188778.572308:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188778.828597:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188779.084298:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188779.340321:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188779.596821:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188779.853340:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188779.899908:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188779.899951:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188779.899984:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188779.900011:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:02020000:0.0:1776188780.108424:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188780.220648:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776188780.220671:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000098305fef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664231808:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776188780.220734:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188780.221112:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000098305fef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664231808:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776188780.225150:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664231872 00000100:00100000:0.0:1776188780.225250:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6321 00000100:00100000:0.0:1776188780.225259:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005ed2a569 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664231872:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776188780.225289:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776188780.225666:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005ed2a569 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664231872:12345-0@lo:13:osp-pre-0-0.0 Request processed in 405us (703us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188780.225674:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6321 00010000:02020000:0.0:1776188780.364287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188780.620318:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188780.881339:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188781.132304:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188781.245857:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d34d173 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664232000:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188781.246125:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188781.246159:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004f9c7e25 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664232448:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188781.246212:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188781.246236:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006cdcc0ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664232704:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776188781.246270:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664232448 00000100:00100000:0.0:1776188781.246310:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8143 00000100:00100000:0.0:1776188781.246316:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003258ba66 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664232448:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188781.246362:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003258ba66 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664232448:12345-0@lo:400:kworker/u8:1.0 Request processed in 45us (150us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188781.246369:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8143 00000100:00100000:0.0:1776188781.246393:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664232000 00000100:00100000:0.0:1776188781.246426:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1219 00000100:00100000:0.0:1776188781.246429:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008da73e5e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664232000:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188781.246456:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008da73e5e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664232000:12345-0@lo:400:kworker/u8:1.0 Request processed in 26us (335us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188781.246463:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1219 00000100:00100000:0.0:1776188781.247035:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664232128 00000100:00100000:0.0:1776188781.247148:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664232256 00000100:00100000:0.0:1776188781.247170:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57014 00000100:00100000:0.0:1776188781.247213:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000b229330 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664232256:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776188781.247221:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776188781.247253:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000b229330 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664232256:12345-0@lo:38:kworker/u8:1.0 Request processed in 40us (243us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188781.247259:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57014 00000100:00100000:0.0:1776188781.247282:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188781.247358:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a635110a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664232512:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188781.247382:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188781.247409:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664232064 00000100:00100000:0.0:1776188781.247434:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664232512 00000100:00100000:0.0:1776188781.247451:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9481 00000100:00100000:0.0:1776188781.247455:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000694ac12a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664232064:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188781.247570:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000694ac12a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664232064:12345-0@lo:400:kworker/u8:1.0 Request processed in 114us (288us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188781.247576:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9481 00000100:00100000:0.0:1776188781.247588:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9482 00000100:00100000:0.0:1776188781.247591:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ac30dfaa pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664232512:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188781.247613:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ac30dfaa pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664232512:12345-0@lo:400:kworker/u8:1.0 Request processed in 21us (232us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188781.247617:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9482 00000100:00100000:0.0:1776188781.247665:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004252561c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664232064:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188781.247687:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a635110a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664232512:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188781.247726:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664232320 00000100:00100000:0.0:1776188781.247747:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9004 00000100:00100000:0.0:1776188781.247751:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003bb8357b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664232320:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776188781.247759:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776188781.247783:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003bb8357b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664232320:12345-0@lo:38:kworker/u8:1.0 Request processed in 32us (726us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188781.247789:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9004 00000100:00100000:0.0:1776188781.247843:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188781.247861:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e01c7a33 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664232576:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188781.247884:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188781.247920:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664232192 00000100:00100000:0.0:1776188781.247941:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9483 00000100:00100000:0.0:1776188781.247945:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007f9bf6ea pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664232192:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188781.247977:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007f9bf6ea pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664232192:12345-0@lo:400:kworker/u8:1.0 Request processed in 31us (134us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188781.247982:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9483 00000100:00100000:0.0:1776188781.248002:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664232576 00000100:00100000:0.0:1776188781.248028:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8144 00000100:00100000:0.0:1776188781.248032:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004abaab4f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664232576:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188781.248374:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004abaab4f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664232576:12345-0@lo:400:kworker/u8:1.0 Request processed in 340us (488us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188781.248381:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8144 00000100:00100000:0.0:1776188781.248409:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc3ee522 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664232192:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188781.248417:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e01c7a33 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664232576:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188781.250612:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188781.250665:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d34d173 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664232000:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188781.250686:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004f9c7e25 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664232448:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188781.250753:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664231936 00000100:00100000:0.0:1776188781.250786:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664232704 00000100:00100000:0.0:1776188781.250807:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6322 00000100:00100000:0.0:1776188781.250813:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e0dcc408 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664231936:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776188781.250829:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776188781.250869:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e0dcc408 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664231936:12345-0@lo:13:osp-pre-1-1.0 Request processed in 56us (6030us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188781.250880:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6322 00000100:00100000:0.0:1776188781.250896:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6323 00000100:00100000:0.0:1776188781.250901:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d781d9c8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664232704:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776188781.250911:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776188781.250958:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d781d9c8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664232704:12345-0@lo:13:osp-pre-0-1.0 Request processed in 57us (347us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188781.250963:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6323 00010000:02020000:0.0:1776188781.388294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188781.644515:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188781.900330:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188782.156410:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188782.412292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188782.668288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188782.924321:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188783.180279:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188783.436291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188783.692288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188783.948324:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188784.204347:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188784.460350:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188784.718125:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188784.972601:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188785.019963:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654926784 00000100:00100000:0.0:1776188785.019997:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8147 00000100:00100000:0.0:1776188785.020002:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000377d6962 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654926784:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776188785.028391:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000377d6962 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654926784:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 8380us (8850us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188785.028407:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8147 00010000:02020000:0.0:1776188785.228310:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188785.342652:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c5a99ce9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664232768:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776188785.342723:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188785.343582:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c5a99ce9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664232768:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776188785.343794:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9e630d1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664232832:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776188786.364881:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776188786.364903:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000079a89d96 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664232896:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776188786.364974:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188786.365011:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9e630d1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664233152:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188786.365048:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188786.365079:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776188786.365086:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000074e02040 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664233344:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188786.365116:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188786.365138:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a829435f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664233536:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188786.365195:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188786.365237:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664233344 00000100:00100000:0.0:1776188786.365284:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664233536 00000100:00100000:0.0:1776188786.365310:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8148 00000100:00100000:0.0:1776188786.365316:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006392c95c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664233344:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188786.365367:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006392c95c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664233344:12345-0@lo:400:kworker/u8:0.0 Request processed in 49us (250us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188786.365376:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8148 00000100:00100000:0.0:1776188786.365398:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8149 00000100:00100000:0.0:1776188786.365405:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000091ef61c6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664233536:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188786.365444:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000091ef61c6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664233536:12345-0@lo:400:kworker/u8:0.0 Request processed in 38us (274us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188786.365455:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8149 00000100:00100000:0.0:1776188786.365482:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664233280 00000100:00100000:0.0:1776188786.365507:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9005 00000100:00100000:0.0:1776188786.365513:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001b0eae72 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664233280:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776188786.365542:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776188786.365577:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001b0eae72 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664233280:12345-0@lo:38:kworker/u8:0.0 Request processed in 65us (275us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188786.365583:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9005 00000100:00100000:0.0:1776188786.365618:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a829435f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664233536:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188786.365628:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000074e02040 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664233344:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188786.365664:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664233152 00000100:00100000:0.0:1776188786.365687:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664233088 00000100:00100000:0.0:1776188786.365698:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664233216 00000100:00100000:0.0:1776188786.365710:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664233024 00000100:00100000:0.0:1776188786.365724:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664233472 00000100:00100000:0.0:1776188786.365740:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9485 00000100:00100000:0.0:1776188786.365744:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c1bd7ae7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664233152:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188786.365783:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c1bd7ae7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664233152:12345-0@lo:400:kworker/u8:0.0 Request processed in 38us (739us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188786.365792:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9485 00000100:00100000:0.0:1776188786.365811:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9486 00000100:00100000:0.0:1776188786.365815:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008950b44b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664233024:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188786.365855:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008950b44b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664233024:12345-0@lo:400:kworker/u8:0.0 Request processed in 37us (475us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188786.365867:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9486 00000100:00100000:0.0:1776188786.365882:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9487 00000100:00100000:0.0:1776188786.365886:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bc8f2779 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664233472:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188786.365928:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bc8f2779 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664233472:12345-0@lo:400:kworker/u8:0.0 Request processed in 40us (492us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188786.365935:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9487 00000100:00100000:0.0:1776188786.366443:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57017 00000100:00100000:0.0:1776188786.366446:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000aa3f364b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664233216:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776188786.366453:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776188786.366473:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000aa3f364b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664233216:12345-0@lo:38:kworker/u8:0.0 Request processed in 27us (1229us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188786.366477:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57017 00000100:00100000:0.0:1776188786.366497:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9e630d1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664233152:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188786.366631:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000079a89d96 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664232896:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776188786.366651:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776188786.366657:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bc3d254c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664233600:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776188786.366683:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188786.366744:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664232960 00000100:00100000:0.0:1776188786.366769:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1221 00000100:00100000:0.0:1776188786.366772:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e858e44a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664232960:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188786.366802:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e858e44a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664232960:12345-0@lo:400:kworker/u8:0.0 Request processed in 29us (1309us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188786.366808:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1221 00000100:00080000:0.0:1776188786.366956:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776188786.366960:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776188786.366964:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c094ed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664233088:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776188786.366972:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776188786.366975:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776188786.366977:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e71929fa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664233216:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776188786.367002:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000022af59eb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664232960:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188786.367011:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b54d6c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664233408:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188786.367018:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000063864031 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664233024:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188786.367029:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002ef57fef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664233472:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188786.367076:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664233600 00000100:00100000:0.0:1776188786.367106:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6327 00000100:00100000:0.0:1776188786.367110:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000079a89d96 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664233600:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776188786.367122:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776188786.367154:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000079a89d96 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664233600:12345-0@lo:13:osp-pre-0-1.0 Request processed in 43us (471us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188786.367160:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6327 00000100:00100000:0.0:1776188786.367212:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bc3d254c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664233600:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776188790.143089:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654926848 00000100:00100000:0.0:1776188790.143724:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1222 00000100:00100000:0.0:1776188790.143730:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ff8d0a76 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654926848:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776188790.143764:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ff8d0a76 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654926848:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 34us (2948us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188790.143771:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1222 00000100:00100000:0.0:1776188790.461191:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bc3d254c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664233728:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776188790.461265:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188790.461808:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bc3d254c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664233728:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776188791.485210:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188791.485242:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664233792 00000100:00100000:3.0:1776188791.485971:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1223 00000100:00100000:3.0:1776188791.485975:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f07038bd pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664233792:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188791.486011:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f07038bd pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664233792:12345-0@lo:400:kworker/u8:0.0 Request processed in 35us (1015us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188791.486016:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1223 00000100:00100000:3.0:1776188791.486047:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c5a99ce9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664233792:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188791.486521:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000098305fef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664233920:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776188791.486555:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188791.486572:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e72d32a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664234048:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776188791.486590:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188791.486602:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b75a9f7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664234112:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776188791.486622:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188791.486669:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664233920 00000100:00100000:0.0:1776188791.486690:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664234048 00000100:00100000:0.0:1776188791.486700:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57019 00000100:00100000:0.0:1776188791.486703:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000046646cdd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664233920:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776188791.486717:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776188791.486749:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000046646cdd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664233920:12345-0@lo:38:kworker/u8:0.0 Request processed in 45us (194us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188791.486754:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57019 00000100:00100000:0.0:1776188791.486765:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57020 00000100:00100000:0.0:1776188791.486768:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001686e875 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664234048:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776188791.486774:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776188791.486792:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001686e875 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664234048:12345-0@lo:38:kworker/u8:0.0 Request processed in 23us (202us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188791.486797:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57020 00000100:00100000:0.0:1776188791.486836:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664234112 00000100:00100000:0.0:1776188791.486855:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9006 00000100:00100000:0.0:1776188791.486859:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bde4c712 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664234112:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776188791.486865:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776188791.486885:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bde4c712 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664234112:12345-0@lo:38:kworker/u8:0.0 Request processed in 26us (264us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188791.486890:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9006 00000100:00080000:0.0:1776188791.486919:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776188791.486923:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776188791.486926:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000098305fef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664233920:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776188791.486934:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776188791.486937:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776188791.486939:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e72d32a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664234048:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776188791.486945:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776188791.486947:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776188791.486960:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b75a9f7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664234112:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776188791.488770:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6331 00000100:00100000:0.0:1776188791.488775:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000db0032a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664234560:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776188791.488786:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776188791.488813:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000db0032a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664234560:12345-0@lo:13:osp-pre-1-1.0 Request processed in 37us (373us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188791.488819:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6331 00000100:00100000:3.0:1776188791.489153:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c4620d7d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664234560:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776188791.489166:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000576bf6bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664234496:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776188795.263201:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654927232 00000100:00100000:0.0:1776188795.263206:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654927168 00000100:00100000:3.0:1776188795.263256:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9492 00000100:00100000:3.0:1776188795.263264:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006145bcbc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654927168:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1776188795.263332:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006145bcbc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654927168:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 67us (304us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188795.263340:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9492 00000100:00100000:3.0:1776188795.264033:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57021 00000100:00100000:3.0:1776188795.264039:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000154c017b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654927232:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:3.0:1776188795.264054:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776188795.264102:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000154c017b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654927232:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 62us (1103us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188795.264108:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57021 00000100:00100000:0.0:1776188795.580544:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776188795.580567:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000576bf6bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664234624:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776188795.580637:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188795.581005:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000576bf6bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664234624:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776188795.581292:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776188795.581302:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000925fa05a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664234688:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776188795.581332:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188795.581683:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000925fa05a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664234688:0@lo:13:osp-pre-0-0.0 00000100:00080000:0.0:1776188796.605423:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776188796.605428:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776188796.605431:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776188796.605437:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 8938 00000100:00080000:0.0:1776188796.605441:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:0.0:1776188796.605446:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776188796.605466:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f9e630d1 x1862459664235008/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776188796.605491:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776188796.605495:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776188796.605496:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776188796.605500:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 8938 00000100:00080000:0.0:1776188796.605502:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00080000:0.0:1776188796.605516:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776188796.605526:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000074e02040 x1862459664235136/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776188796.605533:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776188796.605537:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776188796.605538:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776188796.605541:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 8938 00000100:00080000:0.0:1776188796.605542:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00080000:0.0:1776188796.605545:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776188796.605556:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002ef57fef x1862459664235200/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776188796.605594:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9e630d1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664235008:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776188796.605636:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188796.605656:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000074e02040 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664235136:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776188796.605678:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188796.605689:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002ef57fef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664235200:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776188796.605711:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188796.605744:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664235008 00000100:00100000:0.0:1776188796.605776:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664235136 00000100:00100000:0.0:1776188796.605794:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57022 00000100:00100000:0.0:1776188796.605800:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cc3ee522 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664235008:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776188796.605815:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776188796.605864:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cc3ee522 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664235008:12345-0@lo:38:kworker/u8:1.0 Request processed in 63us (227us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188796.605870:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57022 00000100:00100000:0.0:1776188796.605885:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57023 00000100:00100000:0.0:1776188796.605888:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000c5611bf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664235136:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776188796.605894:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776188796.605916:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000c5611bf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664235136:12345-0@lo:38:kworker/u8:1.0 Request processed in 27us (238us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188796.605921:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57023 00000100:00100000:0.0:1776188796.605938:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664235200 00000100:00100000:0.0:1776188796.605955:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9007 00000100:00100000:0.0:1776188796.605958:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005226aabe pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664235200:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776188796.605964:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776188796.605986:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005226aabe pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664235200:12345-0@lo:38:kworker/u8:1.0 Request processed in 27us (275us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188796.605991:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9007 00000100:00080000:0.0:1776188796.606026:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776188796.606029:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776188796.606033:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9e630d1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664235008:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776188796.606055:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776188796.606057:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776188796.606060:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000074e02040 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664235136:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776188796.606069:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776188796.606071:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776188796.606073:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002ef57fef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664235200:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776188796.610299:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664235328 00000100:00100000:0.0:1776188796.610369:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8158 00000100:00100000:0.0:1776188796.610375:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000960dca04 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664235328:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188796.610467:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000960dca04 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664235328:12345-0@lo:400:kworker/u8:1.0 Request processed in 91us (2845us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188796.610474:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8158 00000100:00100000:0.0:1776188796.611020:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a829435f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664235328:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188796.611879:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e71929fa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664234880:0@lo:400:kworker/u8:1.0 00010000:02020000:0.0:1776188798.797375:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188799.054188:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188799.308471:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188799.564461:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188799.821218:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188800.076289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188800.332297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776188800.382692:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654927552 00000100:00100000:3.0:1776188800.382863:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8159 00000100:00100000:3.0:1776188800.382868:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f435eca pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2280:x1862459654927552:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1776188800.382899:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f435eca pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2280:x1862459654927552:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 30us (907us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188800.382905:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8159 00000100:00100000:0.0:1776188800.383589:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654927424 00000100:00100000:0.0:1776188800.383628:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654927488 00000100:00100000:0.0:1776188800.383646:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9496 00000100:00100000:0.0:1776188800.383650:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f37d44e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654927424:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776188800.383700:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f37d44e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654927424:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 49us (1813us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188800.383854:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9496 00000100:00100000:0.0:1776188800.383871:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57024 00000100:00100000:0.0:1776188800.383875:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000925fa05a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654927488:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:0.0:1776188800.383889:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776188800.383916:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000925fa05a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654927488:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 40us (562us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188800.383922:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57024 00010000:02020000:0.0:1776188800.589362:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188800.700469:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dd5e571c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664235456:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776188800.700538:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188800.701890:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664235456 00000100:00100000:3.0:1776188800.701960:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6336 00000100:00100000:3.0:1776188800.701966:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000dc90dcc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664235456:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776188800.702000:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776188800.702074:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000dc90dcc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664235456:12345-0@lo:13:osp-pre-0-0.0 Request processed in 107us (1535us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188800.702081:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6336 00000100:00100000:0.0:1776188800.704315:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776188800.704327:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a0b45fb4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664235520:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776188800.704378:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188800.704770:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664235520 00000100:00100000:3.0:1776188800.704811:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6337 00000100:00100000:3.0:1776188800.704816:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004252561c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664235520:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776188800.704841:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776188800.704882:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004252561c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664235520:12345-0@lo:13:osp-pre-1-0.0 Request processed in 66us (505us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188800.704889:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6337 00000100:00100000:0.0:1776188800.704958:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a0b45fb4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664235520:0@lo:13:osp-pre-1-0.0 00010000:02020000:0.0:1776188800.844302:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188801.103272:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188801.356263:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188801.615203:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776188801.724497:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000a0e79be pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664235648:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776188801.724570:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188801.724609:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776188801.724618:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dd5e571c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664235584:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188801.724659:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188801.724682:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f4820b3d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664236096:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188801.724726:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188801.724758:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776188801.724765:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000071a19d57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664236032:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188801.724794:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188801.724809:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776188801.724818:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000114609f6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664236288:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776188801.724847:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188801.724884:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664236288 00000100:00100000:3.0:1776188801.724910:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6339 00000100:00100000:3.0:1776188801.724920:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007a0ce2a1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664236288:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776188801.724935:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776188801.724973:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007a0ce2a1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664236288:12345-0@lo:13:osp-pre-0-1.0 Request processed in 54us (130us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188801.724980:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6339 00000100:00100000:3.0:1776188801.725009:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664235584 00000100:00100000:3.0:1776188801.725039:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1227 00000100:00100000:3.0:1776188801.725044:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000afb92d61 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664235584:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188801.725097:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000afb92d61 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664235584:12345-0@lo:400:kworker/u8:0.0 Request processed in 51us (438us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188801.725109:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1227 00000100:00100000:3.0:1776188801.725146:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664236096 00000100:00100000:3.0:1776188801.725169:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664236032 00000100:00100000:3.0:1776188801.725210:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8160 00000100:00100000:3.0:1776188801.725215:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c0bb1b99 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664236096:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188801.725260:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c0bb1b99 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664236096:12345-0@lo:400:kworker/u8:0.0 Request processed in 42us (533us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188801.725269:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8160 00000100:00100000:3.0:1776188801.725285:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8161 00000100:00100000:3.0:1776188801.725290:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003987cbc7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664236032:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188801.725327:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003987cbc7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664236032:12345-0@lo:400:kworker/u8:0.0 Request processed in 36us (533us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188801.725335:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8161 00000100:00100000:3.0:1776188801.725351:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8162 00000100:00100000:3.0:1776188801.725356:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002e574ef8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664236224:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188801.725390:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002e574ef8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664236224:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (97us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188801.725396:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8162 00000100:00100000:3.0:1776188801.725432:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57025 00000100:00100000:3.0:1776188801.725440:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000114609f6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664235776:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776188801.725456:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188801.725499:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000114609f6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664235776:12345-0@lo:38:kworker/u8:0.0 Request processed in 56us (444us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188801.725507:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57025 00000100:00100000:3.0:1776188801.725547:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000098305fef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664235840:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188801.725561:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006cdcc0ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664236224:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188801.725602:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000071a19d57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664236032:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188801.725621:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f4820b3d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664236096:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188801.725634:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dd5e571c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664235584:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188801.726253:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664235968 00000100:00100000:0.0:1776188801.726290:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9008 00000100:00100000:0.0:1776188801.726294:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0b45fb4 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664235968:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776188801.726303:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776188801.726357:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0b45fb4 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664235968:12345-0@lo:38:kworker/u8:0.0 Request processed in 62us (1207us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188801.726364:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9008 00000100:00080000:3.0:1776188801.731012:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188801.731023:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188801.731030:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bc3d254c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664235968:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776188801.733665:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664235712 00000100:00100000:3.0:1776188801.734761:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b75a9f7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664235712:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188801.734775:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001e4997ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664236160:0@lo:400:kworker/u8:0.0 00010000:02020000:0.0:1776188801.871695:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188802.126973:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188802.380382:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188802.636531:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188802.892457:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188803.667263:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188803.924288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188804.180850:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188804.436291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188804.692337:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188804.948289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188805.205866:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776188805.498595:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188805.498633:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188805.498673:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188805.498709:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654927744 00000100:00100000:3.0:1776188805.498747:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654927680 00000100:00100000:3.0:1776188805.498775:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9500 00000100:00100000:3.0:1776188805.498780:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ab0cd203 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654927680:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1776188805.498846:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ab0cd203 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654927680:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 65us (173us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188805.498853:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9500 00000100:00100000:0.0:1776188805.498905:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654927616 00000100:00100000:0.0:1776188805.498935:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1228 00000100:00100000:0.0:1776188805.498939:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008f85e8f7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654927616:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776188805.498966:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008f85e8f7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654927616:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 27us (373us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188805.498973:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1228 00000100:00100000:3.0:1776188805.820602:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004f7a6a77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664236352:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776188805.820680:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188805.824452:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004f7a6a77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664236416:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776188805.824515:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188805.824934:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004f7a6a77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664236416:0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1776188806.844416:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188806.844422:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188806.844425:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188806.844432:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 8948 00000100:00080000:3.0:1776188806.844435:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776188806.844439:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188806.844455:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000a0e79be x1862459664236608/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776188806.844469:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188806.844471:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188806.844472:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188806.844474:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 8948 00000100:00080000:3.0:1776188806.844475:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776188806.844477:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188806.844483:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000dd5e571c x1862459664236736/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776188806.844487:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188806.844489:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188806.844490:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188806.844491:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 8948 00000100:00080000:3.0:1776188806.844492:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776188806.844494:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188806.844502:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a969b6e4 x1862459664236800/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776188806.844540:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c4620d7d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664236672:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188806.844597:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188806.844637:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000880386e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664237056:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188806.844668:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188806.844703:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1776188806.844709:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b15f52c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664236544:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188806.844733:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188806.844748:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3b3d75f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664236992:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188806.844767:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188806.844792:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000a0e79be pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664236608:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188806.844813:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188806.844827:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dd5e571c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664236736:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188806.844847:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188806.844861:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a969b6e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664236800:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188806.844880:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188806.844912:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664236672 00000100:00100000:3.0:1776188806.844951:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664236544 00000100:00100000:3.0:1776188806.844962:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664236992 00000100:00100000:3.0:1776188806.844971:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664236608 00000100:00100000:3.0:1776188806.844979:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664236736 00000100:00100000:3.0:1776188806.844989:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9501 00000100:00100000:3.0:1776188806.844994:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002696d473 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664236672:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188806.845041:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002696d473 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664236672:12345-0@lo:400:kworker/u8:0.0 Request processed in 46us (443us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188806.845048:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9501 00000100:00100000:3.0:1776188806.845065:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9502 00000100:00100000:3.0:1776188806.845068:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000035725ab pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664236544:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188806.845092:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000035725ab pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664236544:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (359us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188806.845097:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9502 00000100:00100000:3.0:1776188806.845107:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9503 00000100:00100000:3.0:1776188806.845110:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000eaf48489 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664236992:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188806.845132:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000eaf48489 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664236992:12345-0@lo:400:kworker/u8:0.0 Request processed in 22us (366us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188806.845137:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9503 00000100:00100000:3.0:1776188806.845147:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57028 00000100:00100000:3.0:1776188806.845150:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ace99b9b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664236608:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776188806.845164:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188806.845225:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ace99b9b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664236608:12345-0@lo:38:kworker/u8:0.0 Request processed in 73us (411us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188806.845230:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57028 00000100:00100000:3.0:1776188806.845239:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57029 00000100:00100000:3.0:1776188806.845242:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000be3c8080 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664236736:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776188806.845249:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188806.845268:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000be3c8080 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664236736:12345-0@lo:38:kworker/u8:0.0 Request processed in 24us (421us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188806.845273:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57029 00000100:00100000:3.0:1776188806.845291:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664237056 00000100:00100000:3.0:1776188806.845313:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8164 00000100:00100000:3.0:1776188806.845317:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fdab0c5c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664237056:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188806.845345:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fdab0c5c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664237056:12345-0@lo:400:kworker/u8:0.0 Request processed in 27us (676us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188806.845350:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8164 00000100:00100000:3.0:1776188806.845381:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664236800 00000100:00100000:3.0:1776188806.845398:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9009 00000100:00100000:3.0:1776188806.845401:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a1abf153 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664236800:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776188806.845410:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188806.845433:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a1abf153 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664236800:12345-0@lo:38:kworker/u8:0.0 Request processed in 30us (552us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188806.845438:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9009 00000100:00080000:3.0:1776188806.845473:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188806.845476:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188806.845479:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000a0e79be pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664236608:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776188806.845487:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188806.845489:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188806.845491:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dd5e571c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664236736:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776188806.845498:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188806.845500:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188806.845502:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a969b6e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664236800:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188806.845534:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b15f52c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664236544:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188806.845542:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3b3d75f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664236992:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188806.845550:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c4620d7d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664236672:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188806.845558:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000880386e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664237056:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188806.845995:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064636c08 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664236480:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188806.846034:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188806.846059:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001b0d11fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664236928:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188806.846080:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188806.846101:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1776188806.846105:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c731e56f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664236864:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188806.846124:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188806.846139:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c921b786 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664237120:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188806.846156:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188806.846208:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664236928 00000100:00100000:0.0:1776188806.846233:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664236864 00000100:00100000:0.0:1776188806.846248:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664237120 00000100:00100000:0.0:1776188806.846262:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8165 00000100:00100000:0.0:1776188806.846265:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005d34d173 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664236928:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188806.846301:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005d34d173 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664236928:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (220us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188806.846306:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8165 00000100:00100000:0.0:1776188806.846319:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8166 00000100:00100000:0.0:1776188806.846322:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a829435f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664236864:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188806.846346:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a829435f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664236864:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (221us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188806.846351:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8166 00000100:00100000:0.0:1776188806.846361:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8167 00000100:00100000:0.0:1776188806.846365:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008b54d6c4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664237120:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188806.846387:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008b54d6c4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664237120:12345-0@lo:400:kworker/u8:0.0 Request processed in 22us (231us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188806.846392:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8167 00000100:00100000:0.0:1776188806.846428:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c731e56f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664236864:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188806.846437:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c921b786 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664237120:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188806.846444:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001b0d11fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664236928:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188806.857168:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664237184 00000100:00100000:3.0:1776188806.857238:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6342 00000100:00100000:3.0:1776188806.857243:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000381790a0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664237184:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776188806.857266:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776188806.857348:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000381790a0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664237184:12345-0@lo:13:osp-pre-1-1.0 Request processed in 104us (2795us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188806.857354:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6342 00000100:00100000:0.0:1776188806.857452:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004d2de401 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664237184:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776188806.860313:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c5a99ce9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664237248:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776188806.860381:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188806.860925:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c5a99ce9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664237248:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776188810.618480:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188810.618626:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654928000 00000100:00100000:0.0:1776188810.618712:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57030 00000100:00100000:0.0:1776188810.618720:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e6117432 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654928000:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:0.0:1776188810.618746:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776188810.618855:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e6117432 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654928000:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 134us (381us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188810.618863:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57030 00000100:00100000:3.0:1776188810.625383:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654927936 00000100:00100000:3.0:1776188810.625442:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9504 00000100:00100000:3.0:1776188810.625450:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e72d32a7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654927936:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1776188810.625549:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e72d32a7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654927936:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 88us (404us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188810.625566:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9504 00000100:00100000:0.0:1776188810.630239:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654927872 00000100:00100000:0.0:1776188810.630327:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1230 00000100:00100000:0.0:1776188810.630333:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d6b85e7e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654927872:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776188810.630411:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d6b85e7e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654927872:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 77us (1070us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188810.630418:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1230 00000100:00100000:3.0:1776188810.945881:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776188810.945903:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000266f5a0c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664237376:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776188810.945973:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188810.947162:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000266f5a0c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664237376:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776188811.965823:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000266f5a0c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664237504:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776188811.965867:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188811.965899:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776188811.965905:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c5a99ce9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664237568:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188811.965933:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188811.966201:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664237568 00000100:00100000:3.0:1776188811.966233:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1231 00000100:00100000:3.0:1776188811.966238:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004c446be7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664237568:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188811.966314:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004c446be7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664237568:12345-0@lo:400:kworker/u8:0.0 Request processed in 64us (369us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188811.966321:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1231 00000100:00100000:3.0:1776188811.966353:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664237504 00000100:00100000:3.0:1776188811.966378:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6347 00000100:00100000:3.0:1776188811.966382:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000562b6cdb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664237504:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776188811.966395:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776188811.966424:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000562b6cdb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664237504:12345-0@lo:13:osp-pre-1-1.0 Request processed in 41us (557us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188811.966432:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6347 00000100:00100000:0.0:1776188811.969134:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001e4997ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664237952:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188811.969200:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188811.969225:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776188811.969230:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a943cb24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664238016:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188811.969251:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188811.969273:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664237952 00000100:00100000:0.0:1776188811.969295:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664238016 00000100:00100000:0.0:1776188811.969323:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8169 00000100:00100000:0.0:1776188811.969326:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002ef57fef pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664237952:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188811.969357:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002ef57fef pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664237952:12345-0@lo:400:kworker/u8:0.0 Request processed in 30us (157us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188811.969362:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8169 00000100:00100000:0.0:1776188811.969373:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8170 00000100:00100000:0.0:1776188811.969376:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000063864031 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664238016:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188811.969395:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000063864031 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664238016:12345-0@lo:400:kworker/u8:0.0 Request processed in 19us (145us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188811.969401:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8170 00000100:00100000:0.0:1776188811.969428:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a943cb24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664238016:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188811.969435:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001e4997ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664237952:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188815.743824:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188815.743876:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188815.743926:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188815.743951:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188816.060674:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776188816.060695:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ccb5377d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664238144:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776188816.060768:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188816.060869:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664238144 00000100:00100000:3.0:1776188816.060923:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6348 00000100:00100000:3.0:1776188816.060929:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022af59eb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664238144:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776188816.060964:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776188816.061018:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022af59eb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664238144:12345-0@lo:13:osp-pre-1-0.0 Request processed in 88us (250us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188816.061025:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6348 00000100:00100000:3.0:1776188816.061774:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664238208 00000100:00100000:3.0:1776188816.061808:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6349 00000100:00100000:3.0:1776188816.061812:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000032e99411 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664238208:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776188816.061830:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776188816.061871:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000032e99411 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664238208:12345-0@lo:13:osp-pre-0-0.0 Request processed in 57us (166us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188816.061877:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6349 00000100:00100000:0.0:1776188817.084794:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776188817.084817:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000014a0e5e7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664238272:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776188817.084880:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188817.085530:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000014a0e5e7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664238272:0@lo:13:osp-pre-1-1.0 00000100:00080000:3.0:1776188817.086298:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188817.086304:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188817.086307:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188817.086313:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 8958 00000100:00080000:3.0:1776188817.086316:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00080000:3.0:1776188817.086320:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776188817.086328:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000014a0e5e7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664238336:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188817.086339:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000266f5a0c x1862459664238464/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776188817.086374:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1776188817.086376:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1776188817.086379:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188817.086382:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1776188817.086412:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776188817.086418:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ccb5377d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664238400:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188817.086451:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188817.086490:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664238336 00000100:00080000:3.0:1776188817.086598:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 8958 00000100:00080000:3.0:1776188817.086601:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00080000:3.0:1776188817.086604:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188817.086622:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000087b82cd7 x1862459664238656/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776188817.086641:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188817.086645:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188817.086646:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188817.086650:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 8958 00000100:00080000:3.0:1776188817.086652:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 6s 00000100:00080000:3.0:1776188817.086655:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188817.086669:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f0b13a19 x1862459664238720/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776188817.086730:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004d2de401 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664238528:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188817.086770:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188817.086793:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000004e60a0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664238912:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188817.086799:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1233 00000100:00100000:0.0:1776188817.086804:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000459e616f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664238336:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188817.086821:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188817.086841:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188817.086846:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000459e616f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664238336:12345-0@lo:400:kworker/u8:1.0 Request processed in 40us (476us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188817.086846:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1776188817.086854:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1233 00000100:00100000:3.0:1776188817.086856:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc7962a4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664238592:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776188817.086863:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000014a0e5e7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664238336:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188817.086888:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188817.086889:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664238784 00000100:00100000:0.0:1776188817.086911:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664238528 00000100:00100000:3.0:1776188817.086912:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b7a82fb0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664238976:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188817.086914:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664238848 00000100:00100000:0.0:1776188817.086929:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664238912 00000100:00100000:1.0:1776188817.086931:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8172 00000100:00100000:1.0:1776188817.086936:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b2d81ea0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664238784:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188817.086942:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188817.086949:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9510 00000100:00100000:0.0:1776188817.086953:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009820cd1e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664238528:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188817.086977:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664238592 00000100:00100000:1.0:1776188817.086981:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b2d81ea0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664238784:12345-0@lo:400:kworker/u8:1.0 Request processed in 43us (225us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188817.086982:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009820cd1e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664238528:12345-0@lo:400:kworker/u8:1.0 Request processed in 28us (215us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188817.086988:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9510 00000100:00100000:1.0:1776188817.086988:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8172 00000100:00100000:0.0:1776188817.087004:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9511 00000100:00100000:1.0:1776188817.087005:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664238976 00000100:00100000:0.0:1776188817.087008:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000067c9d69b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664238912:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188817.087012:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6351 00000100:00100000:3.0:1776188817.087018:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002e9e0e35 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3361:x1862459664238592:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776188817.087029:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8173 00000100:00100000:0.0:1776188817.087032:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000067c9d69b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664238912:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (213us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188817.087034:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000829e500a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664238848:12345-0@lo:400:kworker/u8:1.0 00002000:00000024:3.0:1776188817.087034:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776188817.087039:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9511 00000100:00100000:0.0:1776188817.087060:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8174 00000100:00100000:0.0:1776188817.087064:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000b36a9c1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+6:3361:x1862459664238976:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188817.087602:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000829e500a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664238848:12345-0@lo:400:kworker/u8:1.0 Request processed in 566us (762us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188817.087609:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8173 00000100:00100000:1.0:1776188817.087640:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006306c888 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664238848:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188817.087649:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d995c27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664238784:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188817.087748:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc7962a4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664238592:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776188817.087758:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004d2de401 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664238528:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188817.087767:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000004e60a0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664238912:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188817.087803:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000b36a9c1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+6:3361:x1862459664238976:12345-0@lo:400:kworker/u8:1.0 Request processed in 737us (864us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188817.087810:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8174 00000100:00100000:1.0:1776188817.087815:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b7a82fb0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664238976:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188817.089368:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664238464 00000100:00100000:0.0:1776188817.089451:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664238656 00000100:00100000:0.0:1776188817.089482:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57034 00000100:00100000:0.0:1776188817.089485:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dfad713d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664238464:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776188817.089494:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776188817.089530:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dfad713d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664238464:12345-0@lo:38:kworker/u8:1.0 Request processed in 44us (931us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188817.089536:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57034 00000100:00100000:0.0:1776188817.089547:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57035 00000100:00100000:0.0:1776188817.089550:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a943cb24 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664238656:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776188817.089556:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776188817.089574:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a943cb24 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664238656:12345-0@lo:38:kworker/u8:1.0 Request processed in 23us (912us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188817.089579:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57035 00000100:00100000:3.0:1776188817.091602:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002e9e0e35 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664238592:12345-0@lo:13:osp-pre-0-1.0 Request processed in 4577us (4709us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188817.091616:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6351 00010000:02020000:1.0:1776188817.244993:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188817.500328:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188817.756377:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188818.012818:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188818.268407:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188818.525209:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188818.780295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188819.036269:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188819.292307:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776188819.452383:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188819.548292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188819.804532:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188820.060286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188820.316967:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188820.572290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188820.829671:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188820.859287:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188820.859341:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188820.859376:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188820.859414:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776188820.859416:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654928576 00000100:00100000:0.0:1776188820.859454:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654928448 00000100:00100000:1.0:1776188820.859470:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8175 00000100:00100000:1.0:1776188820.859477:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008e1f2ed1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2279:x1862459654928576:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776188820.859493:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654928512 00000100:00100000:0.0:1776188820.859513:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9512 00000100:00100000:0.0:1776188820.859518:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000074e02040 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654928448:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776188820.859563:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000074e02040 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654928448:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 38us (182us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188820.859563:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008e1f2ed1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2279:x1862459654928576:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 80us (216us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188820.859582:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8175 00000100:00100000:0.0:1776188820.859583:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9512 00000100:00100000:0.0:1776188820.859607:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57036 00000100:00100000:0.0:1776188820.859611:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f9e630d1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654928512:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776188820.859625:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776188820.859682:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654928384 00000100:00100000:3.0:1776188820.859711:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1234 00000100:00100000:3.0:1776188820.859715:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000de62cc6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654928384:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776188820.859765:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f9e630d1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654928512:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 152us (350us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188820.859773:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57036 00000100:00100000:3.0:1776188820.861967:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000de62cc6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654928384:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 2244us (2682us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188820.861982:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1234 00010000:02020000:1.0:1776188821.084305:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188821.180390:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e071bb1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664239040:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776188821.180459:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188821.182303:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664239040 00000100:00100000:3.0:1776188821.182378:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6352 00000100:00100000:3.0:1776188821.182384:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f00e14c6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664239040:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776188821.182417:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776188821.182468:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f00e14c6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664239040:12345-0@lo:13:osp-pre-1-0.0 Request processed in 84us (2010us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188821.182475:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6352 00000100:00100000:0.0:1776188821.182997:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc7962a4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664239104:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776188821.183039:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188821.183095:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e071bb1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664239040:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776188821.183486:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664239104 00000100:00100000:3.0:1776188821.183522:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6353 00000100:00100000:3.0:1776188821.183526:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a4832ec7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664239104:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776188821.183549:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776188821.183606:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a4832ec7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664239104:12345-0@lo:13:osp-pre-0-0.0 Request processed in 78us (567us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188821.183611:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6353 00000100:00100000:0.0:1776188821.183769:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc7962a4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664239104:0@lo:13:osp-pre-0-0.0 00010000:02020000:1.0:1776188821.340283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188821.596851:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188821.852341:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188822.108307:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776188822.204342:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776188822.204365:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003614bde1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664239168:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776188822.204427:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188822.204625:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664239168 00000100:00100000:3.0:1776188822.204627:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664239232 00000100:00080000:0.0:1776188822.204638:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776188822.204644:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776188822.204648:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776188822.204655:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 8964 00000100:00080000:0.0:1776188822.204660:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00080000:0.0:1776188822.204665:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188822.204669:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6354 00000100:00100000:3.0:1776188822.204675:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c5a99ce9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664239232:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776188822.204687:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b43d2cdc x1862459664239424/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00002000:00000024:3.0:1776188822.204695:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776188822.204740:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c5a99ce9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664239232:12345-0@lo:13:osp-pre-1-1.0 Request processed in 64us (136us total) trans 0 rc 0/0 00000100:00080000:0.0:1776188822.204742:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776188822.204747:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776188822.204748:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776188822.204753:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 8964 00000100:00080000:0.0:1776188822.204756:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:0.0:1776188822.204761:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776188822.204776:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000fcf58f4c x1862459664239552/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776188822.204785:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776188822.204788:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776188822.204790:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776188822.204794:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 8964 00000100:00080000:0.0:1776188822.204796:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 11s 00000100:00080000:0.0:1776188822.204798:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776188822.204807:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a0533a16 x1862459664239616/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776188822.204860:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664239360 00000100:00100000:0.0:1776188822.204888:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664239424 00000100:00100000:0.0:1776188822.204907:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9513 00000100:00100000:0.0:1776188822.204911:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009b75a9f7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664239360:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188822.204945:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009b75a9f7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664239360:12345-0@lo:400:kworker/u8:1.0 Request processed in 33us (264us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188822.204950:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9513 00000100:00100000:0.0:1776188822.204967:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57037 00000100:00100000:0.0:1776188822.204970:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000098305fef pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664239424:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776188822.204982:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188822.205005:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6354 00000100:00100000:0.0:1776188822.205010:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000098305fef pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664239424:12345-0@lo:38:kworker/u8:1.0 Request processed in 37us (251us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188822.205017:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57037 00000100:00100000:1.0:1776188822.205036:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6355 00000100:00100000:1.0:1776188822.205043:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000034c47c21 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664239168:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776188822.205044:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000624c000a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664239808:0@lo:400:kworker/u8:1.0 00002000:00000024:1.0:1776188822.205057:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776188822.205074:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188822.205091:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776188822.205096:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1b59934 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664239488:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188822.205119:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188822.205139:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664239808 00000100:00100000:0.0:1776188822.205158:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664239488 00000100:00100000:0.0:1776188822.205195:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9514 00000100:00100000:0.0:1776188822.205198:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000869e3fc2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664239808:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188822.205230:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000869e3fc2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664239808:12345-0@lo:400:kworker/u8:1.0 Request processed in 31us (157us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188822.205236:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9514 00000100:00100000:0.0:1776188822.205247:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9515 00000100:00100000:0.0:1776188822.205251:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000081598856 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664239488:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188822.205276:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000081598856 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664239488:12345-0@lo:400:kworker/u8:1.0 Request processed in 25us (157us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188822.205282:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9515 00000100:00100000:3.0:1776188822.205544:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000005562bb2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664239360:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188822.205560:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e0967c51 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664239232:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776188822.205593:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664239552 00000100:00100000:3.0:1776188822.205609:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57038 00000100:00100000:3.0:1776188822.205613:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000611c527e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664239552:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776188822.205621:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188822.205646:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000611c527e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664239552:12345-0@lo:38:kworker/u8:1.0 Request processed in 32us (90us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188822.205652:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57038 00000100:00100000:0.0:1776188822.205704:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1b59934 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664239488:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188822.205715:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000624c000a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664239808:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188822.205749:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008242b969 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664239680:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188822.205779:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188822.205810:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1776188822.205816:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddb9f539 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664239296:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188822.205840:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188822.205853:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d8f74173 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664239744:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188822.205877:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188822.205912:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664239680 00000100:00100000:0.0:1776188822.205944:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664239744 00000100:00100000:0.0:1776188822.205965:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8176 00000100:00100000:0.0:1776188822.205970:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ce3ffae5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664239680:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188822.206007:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ce3ffae5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664239680:12345-0@lo:400:kworker/u8:1.0 Request processed in 37us (228us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188822.206014:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8176 00000100:00100000:0.0:1776188822.206027:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8177 00000100:00100000:0.0:1776188822.206030:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0c30fb6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664239744:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188822.206054:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0c30fb6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664239744:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (179us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188822.206058:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8177 00000100:00100000:3.0:1776188822.206564:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664239296 00000100:00100000:3.0:1776188822.206594:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1235 00000100:00100000:3.0:1776188822.206599:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000082b19b4e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664239296:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188822.206631:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000082b19b4e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664239296:12345-0@lo:400:kworker/u8:1.0 Request processed in 32us (790us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188822.206640:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1235 00000100:00100000:1.0:1776188822.206739:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000034c47c21 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664239168:12345-0@lo:13:osp-pre-0-1.0 Request processed in 1695us (2312us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188822.206748:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6355 00000100:00100000:0.0:1776188822.206814:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d8f74173 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664239744:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188822.206828:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008242b969 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664239680:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188822.206844:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddb9f539 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664239296:0@lo:400:kworker/u8:1.0 00010000:02020000:1.0:1776188822.364258:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188822.620357:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188822.876591:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188823.132357:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188823.388307:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776188823.548589:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188823.644320:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188823.901418:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188824.156374:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188824.412343:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188824.668249:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188824.924407:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188825.180296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188825.436588:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188825.692250:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188825.948651:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776188825.980872:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188825.980923:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188825.980965:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188825.980972:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654928768 00000100:00100000:3.0:1776188825.981001:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188825.981013:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654928704 00000100:00100000:3.0:1776188825.981032:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654928832 00000100:00100000:0.0:1776188825.981044:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9516 00000100:00100000:0.0:1776188825.981052:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e5ef59b0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654928704:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776188825.981076:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8178 00000100:00100000:3.0:1776188825.981082:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000f61bb34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2278:x1862459654928832:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776188825.981122:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e5ef59b0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654928704:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 68us (121us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188825.981129:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000f61bb34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2278:x1862459654928832:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 38us (156us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188825.981137:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9516 00000100:00100000:3.0:1776188825.981142:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8178 00000100:00100000:0.0:1776188825.981158:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57039 00000100:00100000:0.0:1776188825.981163:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e0967c51 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654928768:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776188825.981224:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776188825.981253:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e0967c51 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654928768:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 89us (388us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188825.981260:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57039 00000100:00100000:1.0:1776188825.982414:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654928640 00000100:00100000:1.0:1776188825.982470:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1236 00000100:00100000:1.0:1776188825.982476:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000055012753 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654928640:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776188825.982536:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000055012753 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654928640:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 58us (1612us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188825.982543:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1236 00010000:02020000:1.0:1776188826.204295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776188826.300958:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000051b336a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664239936:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776188826.305715:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664239872 00000100:00100000:0.0:1776188826.305771:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6356 00000100:00100000:0.0:1776188826.305777:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e3a29c53 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664239872:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776188826.305812:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776188826.305872:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e3a29c53 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664239872:12345-0@lo:13:osp-pre-0-0.0 Request processed in 95us (397us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188826.305879:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6356 00000100:00100000:1.0:1776188826.306670:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188826.307864:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664239936 00000100:00100000:0.0:1776188826.307931:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6357 00000100:00100000:0.0:1776188826.307936:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000852f310a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664239936:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776188826.307964:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776188826.308025:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000852f310a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664239936:12345-0@lo:13:osp-pre-1-0.0 Request processed in 89us (1358us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188826.308032:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6357 00000100:00100000:1.0:1776188826.309874:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000051b336a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664239936:0@lo:13:osp-pre-1-0.0 00010000:02020000:1.0:1776188826.460464:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188826.716310:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188826.972420:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188827.228287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776188827.324445:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003614bde1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664240000:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776188827.324522:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188827.325636:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003614bde1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664240000:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776188827.326882:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664240064 00000100:00100000:0.0:1776188827.326926:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6359 00000100:00100000:0.0:1776188827.326931:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006cdcc0ce pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664240064:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776188827.326951:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776188827.326995:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006cdcc0ce pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664240064:12345-0@lo:13:osp-pre-1-1.0 Request processed in 62us (258us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188827.327105:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6359 00000100:00100000:0.0:1776188827.327147:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ce3ffae5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664240064:0@lo:13:osp-pre-1-1.0 00000100:00080000:3.0:1776188827.328276:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188827.328282:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188827.328286:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188827.328291:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 8969 00000100:00080000:3.0:1776188827.328296:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1776188827.328300:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188827.328320:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ddb9f539 x1862459664240256/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776188827.328340:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188827.328343:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188827.328345:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188827.328348:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 8969 00000100:00080000:3.0:1776188827.328350:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1776188827.328353:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188827.328362:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d8f74173 x1862459664240384/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776188827.328369:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188827.328372:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188827.328373:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188827.328376:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 8969 00000100:00080000:3.0:1776188827.328377:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1776188827.328380:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188827.328389:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a0533a16 x1862459664240448/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776188827.328430:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e071bb1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664240192:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188827.328588:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fcf58f4c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664240512:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188827.328677:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000051b336a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664240128:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188827.330458:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188827.330506:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664240512 00000100:00100000:1.0:1776188827.332127:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8179 00000100:00100000:1.0:1776188827.332143:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002e574ef8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664240512:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188827.332255:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002e574ef8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664240512:12345-0@lo:400:kworker/u8:0.0 Request processed in 115us (1797us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188827.332264:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8179 00000100:00100000:1.0:1776188827.332314:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fcf58f4c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664240512:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188827.332535:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664240448 00000100:00100000:1.0:1776188827.332564:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9013 00000100:00100000:1.0:1776188827.332567:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006392c95c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664240448:12345-0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776188827.332760:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188827.332784:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b43d2cdc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664240576:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188827.332809:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188827.332840:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664240128 00000100:00100000:0.0:1776188827.332867:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1237 00000100:00100000:0.0:1776188827.332870:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bffe4051 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664240128:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188827.333054:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188827.333141:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000624c000a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664240640:0@lo:400:kworker/u8:0.0 00010000:02000000:1.0:1776188827.334148:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776188827.334222:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006392c95c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664240448:12345-0@lo:38:kworker/u8:0.0 Request processed in 1653us (3865us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188827.334230:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9013 00000100:00100000:1.0:1776188827.334268:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664240192 00000100:00100000:1.0:1776188827.334301:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9517 00000100:00100000:1.0:1776188827.334305:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b1b59934 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664240192:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188827.334335:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b1b59934 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664240192:12345-0@lo:400:kworker/u8:0.0 Request processed in 30us (1283us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188827.334340:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9517 00000100:00100000:1.0:1776188827.334353:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57041 00000100:00100000:1.0:1776188827.334357:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004abaab4f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664240384:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776188827.334366:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188827.334370:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188827.336232:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004abaab4f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664240384:12345-0@lo:38:kworker/u8:0.0 Request processed in 1872us (5914us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188827.336240:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57041 00000100:00100000:1.0:1776188827.336304:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664240576 00000100:00100000:1.0:1776188827.336330:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8180 00000100:00100000:1.0:1776188827.336334:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ce3ffae5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664240576:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188827.336379:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ce3ffae5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664240576:12345-0@lo:400:kworker/u8:0.0 Request processed in 44us (3569us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188827.336384:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8180 00000100:00100000:3.0:1776188827.336389:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e071bb1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664240192:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188827.336411:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776188827.336423:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008242b969 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664240320:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188827.336461:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b43d2cdc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664240576:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188827.336464:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188827.336507:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664240320 00000100:00100000:3.0:1776188827.336538:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9519 00000100:00100000:3.0:1776188827.336543:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fcf58f4c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664240320:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188827.336585:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fcf58f4c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664240320:12345-0@lo:400:kworker/u8:0.0 Request processed in 41us (124us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188827.336593:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9519 00000100:00100000:3.0:1776188827.336633:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008242b969 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664240320:0@lo:400:kworker/u8:0.0 00000100:00080000:1.0:1776188827.336656:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776188827.336662:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776188827.336666:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddb9f539 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664240256:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776188827.336676:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776188827.336679:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776188827.336682:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d8f74173 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664240384:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776188827.336691:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776188827.336694:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776188827.336696:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a0533a16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664240448:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776188827.336859:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bffe4051 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664240128:12345-0@lo:400:kworker/u8:0.0 Request processed in 3986us (4098us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188827.336867:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1237 00000100:00100000:1.0:1776188827.337754:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000051b336a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664240128:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188827.338682:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000624c000a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664240640:0@lo:400:kworker/u8:0.0 00010000:02020000:1.0:1776188827.484284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776188827.644667:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188827.746020:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188827.996332:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188828.252368:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188828.508365:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188828.764252:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188829.021435:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188829.276324:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188829.532284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188829.788445:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188830.044834:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188830.302549:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188830.556285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188830.812767:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188831.068274:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776188831.098748:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188831.098797:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188831.098841:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776188831.098971:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654929088 00000100:00100000:1.0:1776188831.099031:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8181 00000100:00100000:1.0:1776188831.099038:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008242b969 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2279:x1862459654929088:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776188831.099123:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008242b969 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2279:x1862459654929088:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 84us (281us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188831.099139:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8181 00000100:00100000:0.0:1776188831.099144:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654928896 00000100:00100000:1.0:1776188831.099211:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1238 00000100:00100000:1.0:1776188831.099218:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002aae1640 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654928896:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776188831.099244:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002aae1640 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654928896:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 27us (447us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188831.099251:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1238 00000100:00100000:0.0:1776188831.103850:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654929024 00000100:00100000:0.0:1776188831.103914:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57042 00000100:00100000:0.0:1776188831.103926:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000091ef61c6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654929024:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776188831.103954:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776188831.104056:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000091ef61c6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654929024:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 128us (448us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188831.104069:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57042 00010000:02020000:1.0:1776188831.324316:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188831.421040:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776188831.421065:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a0533a16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664240704:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776188831.421143:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188831.421579:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776188831.421588:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b2d81ea0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664240768:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776188831.421630:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188831.421938:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664240704 00000100:00100000:3.0:1776188831.421985:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664240768 00000100:00100000:3.0:1776188831.422005:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6360 00000100:00100000:3.0:1776188831.422010:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000043ca7548 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664240704:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776188831.422038:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776188831.422080:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000043ca7548 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664240704:12345-0@lo:13:osp-pre-1-0.0 Request processed in 69us (938us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188831.422098:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6360 00000100:00100000:3.0:1776188831.422114:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6361 00000100:00100000:3.0:1776188831.422118:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000829e500a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664240768:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776188831.422132:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776188831.422157:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000829e500a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664240768:12345-0@lo:13:osp-pre-0-0.0 Request processed in 39us (528us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188831.422162:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6361 00000100:00100000:0.0:1776188831.422334:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a0533a16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664240704:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776188831.422421:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b2d81ea0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664240768:0@lo:13:osp-pre-0-0.0 00010000:02020000:1.0:1776188831.582351:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776188831.740459:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188831.836312:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188832.094327:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188832.348349:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776188832.444442:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000746ef9bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664240832:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776188832.444515:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188832.444796:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664240832 00000100:00100000:0.0:1776188832.444829:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776188832.444841:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004ef5af0c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664240896:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776188832.444847:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6362 00000100:00100000:3.0:1776188832.444865:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b2d81ea0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664240832:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776188832.444874:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:3.0:1776188832.444889:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776188832.444905:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664240896 00000100:00100000:0.0:1776188832.444927:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6363 00000100:00100000:0.0:1776188832.444931:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006c90bb74 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664240896:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776188832.444944:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776188832.444954:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b2d81ea0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664240832:12345-0@lo:13:osp-pre-1-1.0 Request processed in 88us (439us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188832.444962:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6362 00000100:00100000:0.0:1776188832.444985:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006c90bb74 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664240896:12345-0@lo:13:osp-pre-0-1.0 Request processed in 52us (111us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188832.444992:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6363 00000100:00100000:0.0:1776188832.445039:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004ef5af0c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664240896:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776188832.445093:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000746ef9bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664240832:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776188832.446346:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00080000:1.0:1776188832.446405:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776188832.446411:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776188832.446415:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776188832.446424:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 8974 00000100:00080000:1.0:1776188832.446427:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1776188832.446429:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064cbacbf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664241024:0@lo:400:kworker/u8:0.0 00000100:00080000:1.0:1776188832.446434:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776188832.446458:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000381790a0 x1862459664241088/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776188832.446485:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1776188832.446492:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776188832.446493:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776188832.446497:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776188832.446503:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 8974 00000100:00080000:1.0:1776188832.446508:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776188832.446512:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776188832.446526:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004f7a6a77 x1862459664241216/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776188832.446535:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776188832.446541:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776188832.446543:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776188832.446547:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 8974 00000100:00080000:1.0:1776188832.446549:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776188832.446552:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776188832.446564:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003614bde1 x1862459664241280/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776188832.446843:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664241024 00000100:00100000:3.0:1776188832.446866:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664241152 00000100:00100000:3.0:1776188832.446904:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664241088 00000100:00100000:0.0:1776188832.447674:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9522 00000100:00100000:0.0:1776188832.447679:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000746ef9bb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664241024:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188832.447716:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000746ef9bb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664241024:12345-0@lo:400:kworker/u8:0.0 Request processed in 36us (1229us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188832.447722:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9522 00000100:00100000:0.0:1776188832.447807:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064cbacbf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664241024:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188832.447847:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664240960 00000100:00100000:0.0:1776188832.447869:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1239 00000100:00100000:0.0:1776188832.447872:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006f66ddc9 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664240960:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188832.447901:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006f66ddc9 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664240960:12345-0@lo:400:kworker/u8:0.0 Request processed in 28us (529us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188832.447906:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1239 00000100:00100000:0.0:1776188832.447948:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02fb24b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664241344:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188832.447957:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc7962a4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664240960:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188832.449928:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b7a82fb0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664241472:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188832.449988:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188832.450219:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57044 00000100:00100000:1.0:1776188832.450225:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d781d9c8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664241088:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776188832.450241:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776188832.450286:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d781d9c8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664241088:12345-0@lo:38:kworker/u8:0.0 Request processed in 59us (3643us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188832.450293:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57044 00000100:00100000:1.0:1776188832.450395:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b7a82fb0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664241472:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188832.450462:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d92ffa01 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664241408:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188832.450497:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188832.450762:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664241408 00000100:00100000:1.0:1776188832.450799:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8183 00000100:00100000:1.0:1776188832.450803:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d8f74173 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664241408:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188832.450840:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d8f74173 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664241408:12345-0@lo:400:kworker/u8:0.0 Request processed in 36us (344us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188832.450850:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8183 00000100:00100000:3.0:1776188832.451036:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d92ffa01 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664241408:0@lo:400:kworker/u8:0.0 00010000:02020000:1.0:1776188832.606219:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188832.860295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188833.116352:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188833.372701:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188833.628283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188833.884285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188834.140697:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188834.396601:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188834.652298:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188834.908286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188835.164506:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188835.420311:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188835.676849:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776188835.837368:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188835.934210:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188836.192568:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188836.220606:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654929152 00000100:00100000:1.0:1776188836.220641:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654929216 00000100:00100000:0.0:1776188836.220657:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1240 00000100:00100000:0.0:1776188836.220664:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000002b5a0b3 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654929152:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1776188836.220667:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654929280 00000100:00100000:1.0:1776188836.220682:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9524 00000100:00100000:1.0:1776188836.220686:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000381790a0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654929216:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1776188836.220703:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654929344 00000100:00100000:3.0:1776188836.220739:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8184 00000100:00100000:3.0:1776188836.220745:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000058e0433d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2278:x1862459654929344:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776188836.220801:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000002b5a0b3 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654929152:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 136us (407us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188836.220807:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1240 00000100:00100000:3.0:1776188836.222622:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000058e0433d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2278:x1862459654929344:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 1868us (2166us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188836.222636:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8184 00000100:00100000:1.0:1776188836.223316:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000381790a0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654929216:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 1925us (2132us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188836.223328:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9524 00010000:02020000:1.0:1776188836.444437:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776188836.540427:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776188836.540429:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddb9f539 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664241600:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776188836.540446:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b7a82fb0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664241536:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776188836.542209:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188836.542718:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188836.545902:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6365 00000100:00100000:0.0:1776188836.545908:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004ef5af0c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664241600:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776188836.545960:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776188836.546063:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004ef5af0c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664241600:12345-0@lo:13:osp-pre-1-0.0 Request processed in 154us (3348us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188836.546070:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6365 00000100:00100000:0.0:1776188836.546261:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddb9f539 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664241600:0@lo:13:osp-pre-1-0.0 00010000:02020000:1.0:1776188836.700291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188836.958948:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188837.212292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188837.468329:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188837.564463:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776188837.564481:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa4e2605 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664241728:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776188837.564594:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b7a82fb0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664241664:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776188837.564647:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188837.564690:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776188837.564698:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064cbacbf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664241792:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188837.564719:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664241664 00000100:00100000:3.0:1776188837.564733:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188837.564776:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664241792 00000100:00100000:1.0:1776188837.564777:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6366 00000100:00100000:1.0:1776188837.564786:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b43d2cdc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664241664:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776188837.564812:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776188837.564812:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1241 00000100:00100000:3.0:1776188837.564822:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000064f5974 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664241792:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188837.564898:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188837.564921:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003614bde1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664241856:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188837.564945:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188837.564966:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b43d2cdc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664241664:12345-0@lo:13:osp-pre-0-1.0 Request processed in 169us (308us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188837.564974:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776188837.564975:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6366 00000100:00100000:0.0:1776188837.564982:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e50a27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664241984:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188837.564997:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664241728 00000100:00100000:0.0:1776188837.565011:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188837.565020:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6367 00000100:00100000:1.0:1776188837.565025:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b3e4a548 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664241728:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776188837.565036:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776188837.565075:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b3e4a548 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664241728:12345-0@lo:13:osp-pre-1-1.0 Request processed in 49us (182us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188837.565080:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6367 00000100:00100000:0.0:1776188837.565101:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664241856 00000100:00100000:1.0:1776188837.565362:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa4e2605 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664241728:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776188837.565382:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664241984 00000100:00100000:1.0:1776188837.565399:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5f1e324 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664242176:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188837.565410:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9525 00000100:00100000:0.0:1776188837.565416:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000431c2394 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664241856:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188837.565432:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188837.565464:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000431c2394 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664241856:12345-0@lo:400:kworker/u8:1.0 Request processed in 48us (519us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188837.565471:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9525 00000100:00100000:0.0:1776188837.565482:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9526 00000100:00100000:0.0:1776188837.565485:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000314a7eb9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664241984:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188837.565592:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000314a7eb9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664241984:12345-0@lo:400:kworker/u8:1.0 Request processed in 104us (582us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188837.565599:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9526 00000100:00100000:0.0:1776188837.565635:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664242176 00000100:00100000:0.0:1776188837.565656:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8185 00000100:00100000:0.0:1776188837.565660:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000087b82cd7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664242176:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188837.566068:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000064f5974 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664241792:12345-0@lo:400:kworker/u8:1.0 Request processed in 1246us (1335us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188837.566075:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1241 00000100:00100000:3.0:1776188837.566282:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004f7a6a77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664241920:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776188837.566316:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188837.566333:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d97d7d85 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664242048:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776188837.566352:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188837.566366:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bbf4e716 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664242112:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776188837.566389:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188837.566422:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664242112 00000100:00100000:3.0:1776188837.566443:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9015 00000100:00100000:3.0:1776188837.566447:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000051b336a8 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664242112:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776188837.566460:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188837.566492:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000051b336a8 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664242112:12345-0@lo:38:kworker/u8:1.0 Request processed in 45us (103us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188837.566499:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9015 00000100:00080000:3.0:1776188837.566632:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188837.566636:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188837.566640:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bbf4e716 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664242112:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776188837.566772:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664241920 00000100:00100000:3.0:1776188837.566895:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664242048 00000100:00100000:3.0:1776188837.566915:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57046 00000100:00100000:3.0:1776188837.566919:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ddb9f539 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664241920:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776188837.566926:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188837.567036:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ddb9f539 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664241920:12345-0@lo:38:kworker/u8:1.0 Request processed in 116us (719us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188837.567043:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57046 00000100:00100000:3.0:1776188837.567054:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57047 00000100:00100000:3.0:1776188837.567058:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0533a16 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664242048:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776188837.567064:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188837.568541:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0533a16 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664242048:12345-0@lo:38:kworker/u8:1.0 Request processed in 1474us (2180us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188837.568554:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57047 00000100:00080000:3.0:1776188837.568615:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188837.568619:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188837.568623:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004f7a6a77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664241920:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776188837.568634:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188837.568636:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188837.568639:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d97d7d85 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664242048:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776188837.568910:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e50a27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664241984:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188837.568922:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003614bde1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664241856:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188837.569721:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000087b82cd7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664242176:12345-0@lo:400:kworker/u8:1.0 Request processed in 4058us (4287us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188837.569734:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8185 00000100:00100000:3.0:1776188837.570675:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001fc79665 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664242240:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188837.570710:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188837.570772:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664242240 00000100:00100000:0.0:1776188837.570798:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8186 00000100:00100000:0.0:1776188837.570804:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e071bb1b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664242240:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188837.570851:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e071bb1b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664242240:12345-0@lo:400:kworker/u8:1.0 Request processed in 48us (141us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188837.570856:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8186 00000100:00100000:3.0:1776188837.571434:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001fc79665 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664242240:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188837.572469:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064cbacbf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664241792:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188837.572494:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b7a82fb0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664241664:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776188837.572522:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5f1e324 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664242176:0@lo:400:kworker/u8:1.0 00010000:02020000:1.0:1776188837.724404:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188837.981114:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188838.237188:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188838.492565:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188838.748279:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188839.004875:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188839.260346:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188839.517301:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188839.773074:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776188839.935379:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188840.028290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188840.284436:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188840.541791:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188840.797711:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188841.052297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188841.308300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776188841.375394:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188841.375448:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188841.375486:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188841.375553:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776188841.375657:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654929472 00000100:00100000:1.0:1776188841.375714:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654929536 00000100:00100000:1.0:1776188841.375733:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9528 00000100:00100000:1.0:1776188841.375740:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000064cbacbf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654929472:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1776188841.375828:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000064cbacbf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654929472:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 88us (342us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188841.375836:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9528 00000100:00100000:1.0:1776188841.375856:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57048 00000100:00100000:1.0:1776188841.375860:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003614bde1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654929536:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:1.0:1776188841.375882:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776188841.375908:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003614bde1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654929536:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 47us (355us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188841.375914:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57048 00000100:00100000:0.0:1776188841.376419:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654929600 00000100:00100000:0.0:1776188841.376462:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8187 00000100:00100000:0.0:1776188841.376468:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d97d7d85 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2279:x1862459654929600:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776188841.376503:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d97d7d85 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2279:x1862459654929600:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 35us (1056us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188841.376510:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8187 00010000:02020000:1.0:1776188841.564400:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188841.660493:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776188841.660517:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e50a27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664242368:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776188841.660585:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188841.661403:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664242368 00000100:00100000:1.0:1776188841.661456:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6368 00000100:00100000:1.0:1776188841.661463:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002ac4dbc5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664242368:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776188841.661495:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776188841.661558:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004f7a6a77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664242432:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776188841.661558:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002ac4dbc5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664242368:12345-0@lo:13:osp-pre-1-0.0 Request processed in 93us (973us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188841.661570:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6368 00000100:00100000:0.0:1776188841.661600:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188841.661644:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664242432 00000100:00100000:1.0:1776188841.661670:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6369 00000100:00100000:1.0:1776188841.661674:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000029c2d1d5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664242432:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776188841.661689:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776188841.661727:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000029c2d1d5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664242432:12345-0@lo:13:osp-pre-0-0.0 Request processed in 51us (127us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188841.661733:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6369 00000100:00100000:0.0:1776188841.661799:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004f7a6a77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664242432:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776188841.661847:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e50a27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664242368:0@lo:13:osp-pre-1-0.0 00010000:02020000:1.0:1776188841.820373:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188842.076337:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188842.332288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188842.589400:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:0.0:1776188842.684945:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776188842.684953:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776188842.684955:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776188842.684965:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 8984 00000100:00080000:0.0:1776188842.684968:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776188842.684974:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776188842.684994:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000545eaa39 x1862459664242624/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776188842.685014:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776188842.685018:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776188842.685019:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776188842.685022:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 8984 00000100:00080000:0.0:1776188842.685023:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776188842.685026:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776188842.685034:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000aa4e2605 x1862459664242752/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776188842.685040:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776188842.685043:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776188842.685044:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776188842.685047:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 8984 00000100:00080000:0.0:1776188842.685049:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776188842.685051:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776188842.685061:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000db568034 x1862459664242816/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776188842.685110:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000da0d27b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664242688:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188842.685170:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188842.685295:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000545eaa39 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664242624:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188842.685343:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000060733ebe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664242560:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188842.685433:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000071348c99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664242880:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188842.685466:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188842.685496:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004d99e80b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664243136:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188842.685516:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188842.685536:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1776188842.685542:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000054387670 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664242496:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188842.685569:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188842.685582:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000021503dd8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664242944:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188842.685601:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188842.685633:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664242880 00000100:00100000:1.0:1776188842.685669:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664243136 00000100:00100000:1.0:1776188842.685688:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664242944 00000100:00100000:1.0:1776188842.685704:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8188 00000100:00100000:1.0:1776188842.685710:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f0b13a19 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664242880:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188842.685754:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f0b13a19 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664242880:12345-0@lo:400:kworker/u8:0.0 Request processed in 44us (288us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188842.685760:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8188 00000100:00100000:1.0:1776188842.685776:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8189 00000100:00100000:1.0:1776188842.685780:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005d995c27 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664243136:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188842.685805:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005d995c27 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664243136:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (288us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188842.685810:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8189 00000100:00100000:1.0:1776188842.685819:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8190 00000100:00100000:1.0:1776188842.685823:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b7a82fb0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664242944:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188842.685842:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b7a82fb0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664242944:12345-0@lo:400:kworker/u8:0.0 Request processed in 18us (241us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188842.685858:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8190 00000100:00100000:0.0:1776188842.685899:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3185a52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664243072:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188842.685911:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000021503dd8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664242944:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188842.685925:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000071348c99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664242880:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188842.685931:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188842.685940:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004d99e80b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664243136:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188842.685975:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664243072 00000100:00100000:1.0:1776188842.686002:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8191 00000100:00100000:1.0:1776188842.686007:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000960dca04 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664243072:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188842.686028:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776188842.686037:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a829435f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664243200:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776188842.686043:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000960dca04 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664243072:12345-0@lo:400:kworker/u8:0.0 Request processed in 36us (116us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188842.686050:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8191 00000100:00100000:0.0:1776188842.686068:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188842.686079:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3185a52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664243072:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188842.686098:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664243200 00000100:00100000:3.0:1776188842.686226:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664242496 00000100:00100000:3.0:1776188842.686255:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1243 00000100:00100000:3.0:1776188842.686259:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c1f77e46 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664242496:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188842.686300:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c1f77e46 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664242496:12345-0@lo:400:kworker/u8:0.0 Request processed in 40us (730us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188842.686320:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1243 00000100:00100000:3.0:1776188842.686348:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188842.686363:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa4e2605 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664242752:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188842.686383:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188842.686394:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000db568034 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664242816:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188842.686416:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188842.686442:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664242688 00000100:00100000:3.0:1776188842.686465:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664242624 00000100:00100000:3.0:1776188842.686476:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664242752 00000100:00100000:3.0:1776188842.686489:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9529 00000100:00100000:3.0:1776188842.686492:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001a12eabb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664242688:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188842.686520:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001a12eabb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664242688:12345-0@lo:400:kworker/u8:0.0 Request processed in 27us (1352us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188842.686526:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9529 00000100:00100000:3.0:1776188842.686536:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57049 00000100:00100000:3.0:1776188842.686539:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004f7a6a77 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664242624:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776188842.686551:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188842.686579:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004f7a6a77 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664242624:12345-0@lo:38:kworker/u8:0.0 Request processed in 39us (231us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188842.686585:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57049 00000100:00100000:3.0:1776188842.686596:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57050 00000100:00100000:3.0:1776188842.686599:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000005505a39 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664242752:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776188842.686605:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188842.686624:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000005505a39 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664242752:12345-0@lo:38:kworker/u8:0.0 Request processed in 24us (241us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188842.686629:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57050 00000100:00100000:3.0:1776188842.686650:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664242816 00000100:00100000:3.0:1776188842.686666:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9016 00000100:00100000:3.0:1776188842.686669:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008e777483 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664242816:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776188842.686676:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188842.686697:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008e777483 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664242816:12345-0@lo:38:kworker/u8:0.0 Request processed in 27us (280us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188842.686702:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9016 00000100:00080000:3.0:1776188842.686730:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188842.686734:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188842.686737:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000545eaa39 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664242624:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776188842.686744:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188842.686746:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188842.686748:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa4e2605 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664242752:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776188842.686754:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188842.686757:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188842.686759:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000db568034 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664242816:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188842.686787:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188842.686802:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ab3412dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664243008:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188842.686823:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188842.686839:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e50a27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664243264:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776188842.686870:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188842.694107:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000060733ebe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664242560:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188842.694122:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ab3412dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664243008:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188842.694333:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664243264 00000100:00100000:0.0:1776188842.694386:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6370 00000100:00100000:0.0:1776188842.694392:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008b54d6c4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664243200:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776188842.694416:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776188842.694476:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008b54d6c4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664243200:12345-0@lo:13:osp-pre-1-1.0 Request processed in 83us (8410us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188842.694496:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6370 00000100:00100000:0.0:1776188842.694512:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6371 00000100:00100000:0.0:1776188842.694515:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000071b1847 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664243264:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776188842.694526:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776188842.694557:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000071b1847 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664243264:12345-0@lo:13:osp-pre-0-1.0 Request processed in 36us (7682us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188842.694562:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6371 00000100:00100000:3.0:1776188842.696608:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e50a27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664243264:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776188842.696678:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a829435f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664243200:0@lo:13:osp-pre-1-1.0 00010000:02020000:1.0:1776188842.844530:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188843.100283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188843.356381:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188843.614019:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188843.868287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188844.125414:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188844.380289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188844.636341:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188844.894553:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188845.148305:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188845.405770:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188845.660533:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188845.917678:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188846.172296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188846.428550:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188846.458297:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188846.458343:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188846.458384:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188846.458605:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654929728 00000100:00100000:3.0:1776188846.458781:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9532 00000100:00100000:3.0:1776188846.458789:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a829435f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654929728:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776188846.458866:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a829435f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654929728:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 77us (482us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188846.458874:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9532 00000100:00100000:1.0:1776188846.459494:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654929856 00000100:00100000:1.0:1776188846.459529:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8192 00000100:00100000:1.0:1776188846.459534:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002ef57fef pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2278:x1862459654929856:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776188846.459561:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002ef57fef pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2278:x1862459654929856:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 27us (1219us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188846.459567:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8192 00000100:00100000:0.0:1776188846.463589:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654929792 00000100:00100000:0.0:1776188846.463641:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57051 00000100:00100000:0.0:1776188846.463648:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000058c450e1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654929792:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776188846.463667:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776188846.463733:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000058c450e1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654929792:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 85us (243us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188846.463740:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57051 00010000:02020000:1.0:1776188846.684416:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776188846.780733:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e50a27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664243392:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776188846.780826:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188846.781249:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776188846.781260:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000b36a9c1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664243328:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776188846.781368:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188846.781432:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664243392 00000100:00100000:1.0:1776188846.781475:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664243328 00000100:00100000:1.0:1776188846.781496:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6372 00000100:00100000:1.0:1776188846.781501:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000db568034 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664243392:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776188846.781531:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776188846.781567:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000db568034 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664243392:12345-0@lo:13:osp-pre-1-0.0 Request processed in 65us (741us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188846.781574:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6372 00000100:00100000:1.0:1776188846.781591:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6373 00000100:00100000:1.0:1776188846.781594:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ab3412dc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664243328:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776188846.781610:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776188846.781642:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ab3412dc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664243328:12345-0@lo:13:osp-pre-0-0.0 Request processed in 46us (274us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188846.781648:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6373 00000100:00100000:3.0:1776188846.781666:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e50a27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664243392:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776188846.781684:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000b36a9c1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664243328:0@lo:13:osp-pre-0-0.0 00010000:02020000:1.0:1776188846.940426:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188847.197062:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188847.452246:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188847.708490:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:0.0:1776188847.804428:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776188847.804435:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776188847.804437:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776188847.804445:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 8989 00000100:00080000:0.0:1776188847.804450:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00080000:0.0:1776188847.804455:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776188847.804474:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008e1f2ed1 x1862459664243584/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776188847.804510:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776188847.804513:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776188847.804514:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776188847.804517:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 8989 00000100:00080000:0.0:1776188847.804519:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00080000:0.0:1776188847.804522:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776188847.804531:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005d34d173 x1862459664243712/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776188847.804538:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776188847.804541:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776188847.804542:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776188847.804544:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 8989 00000100:00080000:0.0:1776188847.804546:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 6s 00000100:00080000:0.0:1776188847.804548:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776188847.804556:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005ed2a569 x1862459664243776/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776188847.804610:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c30e127a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664243648:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188847.804678:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188847.804706:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c921b786 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664244032:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188847.804730:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188847.804768:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1776188847.804773:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000063864031 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664243520:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188847.804795:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188847.804809:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001b0d11fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664243968:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188847.804828:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188847.804860:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664243648 00000100:00100000:0.0:1776188847.804895:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664243520 00000100:00100000:0.0:1776188847.804907:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664243968 00000100:00100000:0.0:1776188847.804924:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9533 00000100:00100000:0.0:1776188847.804929:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000880386e2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664243648:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188847.804974:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000880386e2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664243648:12345-0@lo:400:kworker/u8:0.0 Request processed in 44us (297us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188847.804980:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9533 00000100:00100000:0.0:1776188847.804997:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9534 00000100:00100000:0.0:1776188847.805000:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a969b6e4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664243520:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188847.805025:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a969b6e4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664243520:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (230us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188847.805030:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9534 00000100:00100000:0.0:1776188847.805040:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9535 00000100:00100000:0.0:1776188847.805043:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d45cc7cd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664243968:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188847.805065:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d45cc7cd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664243968:12345-0@lo:400:kworker/u8:0.0 Request processed in 22us (237us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188847.805074:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9535 00000100:00100000:0.0:1776188847.805093:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664244032 00000100:00100000:0.0:1776188847.805114:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8193 00000100:00100000:0.0:1776188847.805117:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b3b3d75f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664244032:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188847.805143:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b3b3d75f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664244032:12345-0@lo:400:kworker/u8:0.0 Request processed in 25us (413us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188847.805149:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8193 00000100:00100000:0.0:1776188847.805220:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000063864031 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664243520:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188847.805230:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001b0d11fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664243968:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188847.805238:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c30e127a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664243648:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188847.805245:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c921b786 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664244032:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188847.806155:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e1f2ed1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664243584:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188847.806205:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188847.806223:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d34d173 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664243712:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188847.806243:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188847.806257:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005ed2a569 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664243776:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188847.806278:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188847.806339:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664243776 00000100:00100000:3.0:1776188847.806360:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9017 00000100:00100000:3.0:1776188847.806365:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000054387670 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664243776:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776188847.806378:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188847.806408:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000054387670 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664243776:12345-0@lo:38:kworker/u8:0.0 Request processed in 43us (130us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188847.806414:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9017 00000100:00100000:3.0:1776188847.806427:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776188847.806434:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000da0d27b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664244160:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776188847.806458:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188847.806484:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664243584 00000100:00100000:3.0:1776188847.806501:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664243712 00000100:00100000:3.0:1776188847.806514:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57052 00000100:00100000:3.0:1776188847.806517:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000094e50a27 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664243584:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776188847.806523:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188847.806543:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000094e50a27 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664243584:12345-0@lo:38:kworker/u8:0.0 Request processed in 26us (340us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188847.806548:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57052 00000100:00100000:3.0:1776188847.806557:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57053 00000100:00100000:3.0:1776188847.806560:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000060733ebe pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664243712:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776188847.806565:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188847.806579:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000060733ebe pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664243712:12345-0@lo:38:kworker/u8:0.0 Request processed in 19us (337us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188847.806584:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57053 00000100:00080000:3.0:1776188847.806609:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188847.806612:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188847.806615:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e1f2ed1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664243584:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776188847.806623:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188847.806625:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188847.806627:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d34d173 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664243712:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776188847.806634:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188847.806636:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188847.806638:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005ed2a569 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664243776:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188847.806672:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664244160 00000100:00100000:3.0:1776188847.806707:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6374 00000100:00100000:3.0:1776188847.806711:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000aa4e2605 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664244160:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776188847.806725:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776188847.806766:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000aa4e2605 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664244160:12345-0@lo:13:osp-pre-0-1.0 Request processed in 54us (308us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188847.806772:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6374 00000100:00100000:3.0:1776188847.806798:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000da0d27b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664244160:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776188847.806933:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000576bf6bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664244224:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776188847.806961:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188847.807123:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664243456 00000100:00100000:1.0:1776188847.807221:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000576bf6bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664244224:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776188847.809343:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1245 00000100:00100000:0.0:1776188847.809349:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000353f317b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664243456:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188847.809433:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000353f317b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664243456:12345-0@lo:400:kworker/u8:0.0 Request processed in 83us (3965us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188847.809440:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1245 00000100:00100000:3.0:1776188847.812329:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000b36a9c1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664243456:0@lo:400:kworker/u8:0.0 00010000:02020000:1.0:1776188847.964293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188848.220300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188848.476546:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188848.733378:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188848.989228:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188849.244285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188849.500293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188849.756931:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188850.012326:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188850.270101:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188850.524283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188850.780336:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188851.036354:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188851.293845:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188851.548319:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776188851.578339:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188851.578426:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188851.578462:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188851.578499:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776188851.578567:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654930112 00000100:00100000:1.0:1776188851.578634:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8197 00000100:00100000:1.0:1776188851.578645:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000545eaa39 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2278:x1862459654930112:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1776188851.579699:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000545eaa39 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2278:x1862459654930112:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 1052us (1233us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188851.579710:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8197 00000100:00100000:0.0:1776188851.580331:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654929984 00000100:00100000:0.0:1776188851.580377:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654930048 00000100:00100000:0.0:1776188851.580398:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9536 00000100:00100000:0.0:1776188851.580403:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000da0d27b2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654929984:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776188851.580450:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000da0d27b2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654929984:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 46us (2119us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188851.580460:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9536 00000100:00100000:0.0:1776188851.580476:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57054 00000100:00100000:0.0:1776188851.580480:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000b36a9c1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654930048:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:0.0:1776188851.580495:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776188851.580520:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000b36a9c1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654930048:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 39us (2022us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188851.580526:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57054 00010000:02020000:1.0:1776188851.804322:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188851.901129:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae1dab54 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664244288:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776188851.901279:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188851.903516:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae1dab54 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664244288:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776188851.904376:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005f435eca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664244352:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776188851.904425:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188851.904605:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664244352 00000100:00100000:0.0:1776188851.904660:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6377 00000100:00100000:0.0:1776188851.904665:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000032f89d8f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664244352:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776188851.904689:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776188851.904724:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000032f89d8f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664244352:12345-0@lo:13:osp-pre-0-0.0 Request processed in 58us (299us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188851.904731:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6377 00000100:00100000:3.0:1776188851.904851:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f435eca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664244352:0@lo:13:osp-pre-0-0.0 00010000:02020000:1.0:1776188852.061545:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188852.316428:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188852.573677:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188852.828247:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776188852.924886:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776188852.924909:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000db0032a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664244416:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776188852.924978:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188852.925384:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000db0032a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664244416:0@lo:13:osp-pre-0-1.0 00000100:00080000:1.0:1776188852.929074:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776188852.929081:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776188852.929084:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776188852.929091:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 8994 00000100:00080000:1.0:1776188852.929095:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776188852.929100:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776188852.929118:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000021503dd8 x1862459664244608/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776188852.929148:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776188852.929151:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776188852.929152:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776188852.929155:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 8994 00000100:00080000:1.0:1776188852.929170:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776188852.929193:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776188852.929212:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b09edf48 x1862459664244736/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776188852.929219:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776188852.929222:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776188852.929223:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776188852.929226:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 8994 00000100:00080000:1.0:1776188852.929228:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776188852.929230:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776188852.929253:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000004e60a0f x1862459664244800/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776188852.929314:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000071348c99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664244544:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188852.929380:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188852.929407:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d9b9c94e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664244992:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188852.929436:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188852.929462:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1776188852.929468:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004d99e80b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664244480:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188852.929497:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188852.929513:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fdab0c5c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664244928:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188852.929534:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188852.929572:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000021503dd8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664244608:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776188852.929598:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188852.929617:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b09edf48 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664244736:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776188852.929651:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188852.929668:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000004e60a0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664244800:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776188852.929685:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664244544 00000100:00100000:1.0:1776188852.929693:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188852.929730:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664244992 00000100:00100000:1.0:1776188852.929734:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664244608 00000100:00100000:3.0:1776188852.929752:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664244800 00000100:00100000:0.0:1776188852.929754:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664244736 00000100:00100000:1.0:1776188852.929758:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9537 00000100:00100000:1.0:1776188852.929764:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004d2de401 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664244544:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188852.929776:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9538 00000100:00100000:3.0:1776188852.929779:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9018 00000100:00100000:0.0:1776188852.929785:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000014a0e5e7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664244992:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188852.929786:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b7fee202 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664244800:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776188852.929805:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776188852.929811:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004d2de401 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664244544:12345-0@lo:400:kworker/u8:1.0 Request processed in 46us (430us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188852.929819:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9537 00000100:00100000:1.0:1776188852.929843:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57055 00000100:00100000:1.0:1776188852.929847:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000067bb60df pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664244608:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776188852.929851:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000014a0e5e7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664244992:12345-0@lo:400:kworker/u8:1.0 Request processed in 65us (412us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188852.929862:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9538 00010000:02000000:1.0:1776188852.929863:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776188852.929879:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57056 00000100:00100000:0.0:1776188852.929884:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001ca51788 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664244736:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776188852.929893:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776188852.929896:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000067bb60df pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664244608:12345-0@lo:38:kworker/u8:1.0 Request processed in 48us (296us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188852.929903:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57055 00000100:00100000:0.0:1776188852.929942:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001ca51788 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664244736:12345-0@lo:38:kworker/u8:1.0 Request processed in 56us (291us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188852.929945:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000071348c99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664244544:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188852.929953:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57056 00000100:00100000:1.0:1776188852.929961:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d9b9c94e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664244992:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188852.930002:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006306c888 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664244864:0@lo:400:kworker/u8:1.0 00000100:00080000:1.0:1776188852.930012:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776188852.930016:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776188852.930020:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000021503dd8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664244608:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776188852.930033:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776188852.930037:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776188852.930038:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188852.930042:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b7fee202 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664244800:12345-0@lo:38:kworker/u8:1.0 Request processed in 254us (347us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188852.930043:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b09edf48 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664244736:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776188852.930055:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9018 00000100:00080000:1.0:1776188852.930061:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:0.0:1776188852.930065:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064bc62e8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664245120:0@lo:13:osp-pre-1-1.0 00000100:00080000:1.0:1776188852.930065:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776188852.930071:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000004e60a0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664244800:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776188852.930094:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188852.930110:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664244928 00000100:00100000:3.0:1776188852.930142:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664244864 00000100:00100000:1.0:1776188852.930152:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8198 00000100:00100000:1.0:1776188852.930157:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ccb5377d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664244928:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188852.930213:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ccb5377d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664244928:12345-0@lo:400:kworker/u8:1.0 Request processed in 55us (679us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188852.930312:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1776188852.930319:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cedf4ce4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664244672:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188852.930345:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188852.930358:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000266f5a0c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664245056:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188852.930375:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188852.930397:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664244672 00000100:00100000:0.0:1776188852.930418:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9539 00000100:00100000:0.0:1776188852.930422:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000db0032a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664244672:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188852.930529:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000db0032a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664244672:12345-0@lo:400:kworker/u8:1.0 Request processed in 107us (183us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188852.930536:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9539 00000100:00100000:0.0:1776188852.930568:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cedf4ce4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664244672:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188852.930589:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664245120 00000100:00100000:0.0:1776188852.930611:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6379 00000100:00100000:0.0:1776188852.930615:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd5e571c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664245120:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776188852.930640:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776188852.930673:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd5e571c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664245120:12345-0@lo:13:osp-pre-1-1.0 Request processed in 56us (579us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188852.930678:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6379 00000100:00100000:0.0:1776188852.930723:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064bc62e8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664245120:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776188852.930762:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664245056 00000100:00100000:3.0:1776188852.930784:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8199 00000100:00100000:3.0:1776188852.930789:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000088c2a62e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664244864:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188852.930976:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fdab0c5c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664244928:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188852.930976:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000088c2a62e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664244864:12345-0@lo:400:kworker/u8:1.0 Request processed in 184us (939us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188852.930988:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8199 00000100:00100000:3.0:1776188852.931003:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8200 00000100:00100000:3.0:1776188852.931007:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f435eca pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664245056:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188852.931033:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f435eca pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664245056:12345-0@lo:400:kworker/u8:1.0 Request processed in 25us (657us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188852.931038:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8200 00000100:00100000:3.0:1776188852.931275:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000266f5a0c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664245056:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188852.931287:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006306c888 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664244864:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188852.931717:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8198 00010000:02020000:1.0:1776188853.084295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188853.342727:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188853.596283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188853.852293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188854.108832:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188854.364326:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188854.620280:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188854.878091:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188855.132624:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188855.388270:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188855.644678:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188855.900285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188856.156969:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188856.412295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188856.668306:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188856.703469:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188856.703524:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188856.703569:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188856.703606:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188856.703643:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654930304 00000100:00100000:0.0:1776188856.703677:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654930240 00000100:00100000:0.0:1776188856.703709:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9540 00000100:00100000:0.0:1776188856.703717:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005d34d173 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654930240:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776188856.703786:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005d34d173 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654930240:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 68us (179us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188856.703794:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9540 00000100:00100000:0.0:1776188856.703815:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57057 00000100:00100000:0.0:1776188856.703819:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000064bc62e8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654930304:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776188856.703833:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776188856.703858:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000064bc62e8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654930304:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 37us (396us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188856.703865:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57057 00000100:00100000:1.0:1776188856.703949:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654930176 00000100:00100000:1.0:1776188856.703990:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1248 00000100:00100000:1.0:1776188856.703995:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001aaffad0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654930176:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776188856.704038:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001aaffad0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654930176:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 42us (514us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188856.704044:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1248 00010000:02020000:1.0:1776188856.924302:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776188857.021893:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000013c0eaff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664245248:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776188857.021984:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188857.026238:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664245248 00000100:00100000:3.0:1776188857.026447:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664245184 00000100:00100000:3.0:1776188857.026485:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6380 00000100:00100000:3.0:1776188857.026492:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000006dfa37f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664245248:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776188857.026558:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776188857.026667:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000006dfa37f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664245248:12345-0@lo:13:osp-pre-1-0.0 Request processed in 174us (4683us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188857.027452:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6380 00010000:02020000:1.0:1776188857.180336:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188857.436596:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188857.692248:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188857.948287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:1.0:1776188858.044397:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1776188858.044402:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000dc90dcc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664245376:0@lo:13:osp-pre-1-1.0 00000100:00080000:1.0:1776188858.044405:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776188858.044410:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1776188858.044410:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c0bb1b99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664245312:0@lo:400:kworker/u8:0.0 00000100:00080000:1.0:1776188858.044420:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 8999 00000100:00080000:1.0:1776188858.044425:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:1.0:1776188858.044431:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776188858.044453:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b64538ad x1862459664245568/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776188858.044476:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188858.044481:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776188858.044486:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776188858.044491:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776188858.044493:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776188858.044497:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 8999 00000100:00080000:1.0:1776188858.044500:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00080000:1.0:1776188858.044507:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776188858.044527:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d82e179d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664245632:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188858.044532:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007a0ce2a1 x1862459664245696/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776188858.044542:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776188858.044546:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1776188858.044547:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664245312 00000100:00080000:1.0:1776188858.044550:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776188858.044554:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 8999 00000100:00080000:1.0:1776188858.044557:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00100000:0.0:1776188858.044558:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776188858.044563:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776188858.044578:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f1c366f3 x1862459664245760/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776188858.044592:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1249 00000100:00100000:3.0:1776188858.044599:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000439c1468 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664245312:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188858.044607:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664245632 00000100:00100000:1.0:1776188858.044646:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064636c08 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664245888:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188858.044684:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188858.044718:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b64538ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664245568:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188858.044737:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000439c1468 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664245312:12345-0@lo:400:kworker/u8:0.0 Request processed in 137us (258us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188858.044746:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1249 00000100:00100000:1.0:1776188858.044758:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664245888 00000100:00100000:1.0:1776188858.044794:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8202 00000100:00100000:1.0:1776188858.044801:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b15f52c9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664245888:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188858.044802:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664245376 00000100:00100000:3.0:1776188858.044836:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664245440 00000100:00100000:1.0:1776188858.044847:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b15f52c9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664245888:12345-0@lo:400:kworker/u8:0.0 Request processed in 46us (164us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188858.044855:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8202 00000100:00100000:3.0:1776188858.044866:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6382 00000100:00100000:3.0:1776188858.044873:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c30e127a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664245376:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776188858.044888:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776188858.044923:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c30e127a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664245376:12345-0@lo:13:osp-pre-1-1.0 Request processed in 50us (449us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188858.044931:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6382 00000100:00100000:3.0:1776188858.044948:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6383 00000100:00100000:3.0:1776188858.044952:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b5f1e324 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664245440:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776188858.044963:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776188858.044994:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b5f1e324 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664245440:12345-0@lo:13:osp-pre-0-1.0 Request processed in 41us (494us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188858.044999:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6383 00000100:00100000:3.0:1776188858.045056:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188858.045068:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a0ce2a1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664245696:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188858.045094:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188858.045103:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f1c366f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664245760:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188858.045125:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188858.045165:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664245760 00000100:00100000:3.0:1776188858.045203:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9019 00000100:00100000:3.0:1776188858.045207:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000060a8344d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664245760:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776188858.045221:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188858.045254:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000060a8344d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664245760:12345-0@lo:38:kworker/u8:0.0 Request processed in 46us (129us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188858.045259:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9019 00000100:00080000:3.0:1776188858.045288:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188858.045291:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188858.045294:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f1c366f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664245760:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188858.045334:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000013c0eaff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664245440:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776188858.045363:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664245568 00000100:00100000:1.0:1776188858.048904:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9542 00000100:00100000:1.0:1776188858.049494:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001fc79665 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664245504:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188858.049616:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001fc79665 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664245504:12345-0@lo:400:kworker/u8:0.0 Request processed in 129us (4978us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188858.049862:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9542 00000100:00100000:1.0:1776188858.049894:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57058 00000100:00100000:1.0:1776188858.049900:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f6d90a77 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664245696:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776188858.049915:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776188858.049956:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f6d90a77 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664245696:12345-0@lo:38:kworker/u8:0.0 Request processed in 57us (4862us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188858.049963:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57058 00000100:00080000:1.0:1776188858.050009:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776188858.050014:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776188858.050022:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a0ce2a1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664245696:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776188858.050059:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c4620d7d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664245952:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188858.050084:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003987cbc7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664245504:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188858.050149:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d82e179d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664245632:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188858.050166:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000dc90dcc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664245376:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776188858.050202:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776188858.050211:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d2b21909 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664245824:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188858.050257:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188858.050285:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664245824 00000100:00100000:1.0:1776188858.050314:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8203 00000100:00100000:1.0:1776188858.050318:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d82e179d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664245824:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188858.050353:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d82e179d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664245824:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (96us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188858.050359:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8203 00000100:00100000:1.0:1776188858.050388:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d2b21909 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664245824:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188858.051467:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57059 00000100:00100000:0.0:1776188858.051475:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ae1dab54 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664245568:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776188858.051489:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776188858.051564:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ae1dab54 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664245568:12345-0@lo:38:kworker/u8:0.0 Request processed in 89us (6508us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188858.051570:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57059 00000100:00080000:3.0:1776188858.051909:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188858.051913:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188858.051918:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b64538ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664245568:0@lo:38:kworker/u8:0.0 00010000:02020000:1.0:1776188858.204339:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188858.460283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188858.717673:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188858.972316:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188859.228288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188859.484292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188859.741683:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188859.996378:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188860.252355:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188860.508280:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188860.764265:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188861.020386:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188861.276376:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188861.536752:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188861.788291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188861.818112:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188861.818166:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188861.818237:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188861.818281:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654930432 00000100:00100000:3.0:1776188861.818388:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1250 00000100:00100000:3.0:1776188861.818398:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006339443e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654930432:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776188861.818471:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006339443e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654930432:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 73us (305us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188861.818478:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1250 00000100:00100000:1.0:1776188861.819210:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654930496 00000100:00100000:1.0:1776188861.819250:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9544 00000100:00100000:1.0:1776188861.819255:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000dc90dcc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654930496:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776188861.820215:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000dc90dcc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654930496:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 957us (2109us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188861.820225:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9544 00000100:00100000:0.0:1776188861.826818:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776188861.827306:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654930560 00000100:00100000:1.0:1776188861.827360:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57060 00000100:00100000:1.0:1776188861.827366:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c921b786 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654930560:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776188861.827384:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776188861.827458:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c921b786 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654930560:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 91us (649us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188861.827465:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57060 00010000:02020000:1.0:1776188862.044401:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188862.142708:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664246016 00000100:00100000:0.0:1776188862.142764:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6384 00000100:00100000:0.0:1776188862.142771:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c8b239a6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664246016:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776188862.142801:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776188862.142858:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c8b239a6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664246016:12345-0@lo:13:osp-pre-0-0.0 Request processed in 87us (298us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188862.142865:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6384 00000100:00100000:1.0:1776188862.142957:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001b0d11fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664246016:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776188862.142977:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003b5d789a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664246080:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776188862.143003:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:02020000:1.0:1776188862.300687:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188862.556965:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188862.812469:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188863.069358:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776188863.164649:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f34aeb18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664246208:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188863.164700:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005ee0b9e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664246336:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188863.164725:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188863.164923:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000023a5b7b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664246144:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188863.165344:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001b0d11fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664246656:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776188863.165382:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188863.165439:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664246208 00000100:00100000:1.0:1776188863.165483:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9545 00000100:00100000:1.0:1776188863.165489:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003987cbc7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664246208:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188863.165949:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003987cbc7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664246208:12345-0@lo:400:kworker/u8:0.0 Request processed in 458us (1228us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188863.165956:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9545 00000100:00100000:1.0:1776188863.165980:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664246656 00000100:00100000:1.0:1776188863.166004:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6386 00000100:00100000:1.0:1776188863.166008:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c4620d7d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664246656:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776188863.166022:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776188863.168010:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c4620d7d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664246656:12345-0@lo:13:osp-pre-1-1.0 Request processed in 2000us (2630us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188863.168019:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6386 00000100:00100000:0.0:1776188863.171780:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188863.171829:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d262f767 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664246720:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188863.171895:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188863.172043:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188863.172072:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f92d37ee pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664246592:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188863.172101:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188863.172121:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005e3f10b7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664246848:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188863.172142:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188863.172196:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664246592 00000100:00100000:0.0:1776188863.172200:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664246144 00000100:00100000:3.0:1776188863.172236:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664246848 00000100:00100000:0.0:1776188863.172250:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1251 00000100:00100000:0.0:1776188863.172256:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a3281d23 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664246144:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188863.172272:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8207 00000100:00100000:3.0:1776188863.172279:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e6665a35 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664246592:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188863.172310:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a3281d23 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664246144:12345-0@lo:400:kworker/u8:0.0 Request processed in 52us (267us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188863.172318:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1251 00000100:00100000:3.0:1776188863.172340:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e6665a35 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664246592:12345-0@lo:400:kworker/u8:0.0 Request processed in 59us (240us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188863.172348:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8207 00000100:00100000:3.0:1776188863.172367:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8208 00000100:00100000:3.0:1776188863.172372:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000031aca57e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664246848:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188863.172378:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000023a5b7b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664246144:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188863.172415:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f92d37ee pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664246592:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188863.172426:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000031aca57e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664246848:12345-0@lo:400:kworker/u8:0.0 Request processed in 50us (279us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188863.172430:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005e3f10b7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664246848:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188863.172436:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8208 00000100:00100000:1.0:1776188863.172614:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664246336 00000100:00100000:1.0:1776188863.172879:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664246720 00000100:00100000:1.0:1776188863.172905:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9546 00000100:00100000:1.0:1776188863.172910:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001b0d11fc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664246336:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188863.172952:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001b0d11fc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664246336:12345-0@lo:400:kworker/u8:0.0 Request processed in 41us (1178us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188863.172958:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9546 00000100:00100000:1.0:1776188863.172972:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9547 00000100:00100000:1.0:1776188863.172977:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008e1f2ed1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664246720:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188863.173001:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008e1f2ed1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664246720:12345-0@lo:400:kworker/u8:0.0 Request processed in 25us (1107us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188863.173007:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9547 00000100:00100000:1.0:1776188863.173053:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005ee0b9e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664246336:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188863.173063:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d262f767 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664246720:0@lo:400:kworker/u8:0.0 00010000:02020000:1.0:1776188863.324386:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188863.580390:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188863.836295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188864.092283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188864.348284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188864.604295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188864.860338:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188865.116545:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188865.372287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188865.628620:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188865.884635:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188866.140296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188866.396565:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188866.652578:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188866.908569:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776188866.937992:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188866.938042:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776188866.938164:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654930752 00000100:00100000:0.0:1776188866.938206:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654930688 00000100:00100000:0.0:1776188866.938256:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1252 00000100:00100000:0.0:1776188866.938263:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b7303d3d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654930688:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776188866.938352:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b7303d3d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654930688:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 89us (369us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188866.938361:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1252 00000100:00100000:1.0:1776188866.938365:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9548 00000100:00100000:1.0:1776188866.938371:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0146fc7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654930752:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1776188866.938416:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0146fc7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654930752:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 45us (374us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188866.938424:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9548 00000100:00100000:0.0:1776188866.941972:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188866.942054:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776188866.942220:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654930816 00000100:00100000:3.0:1776188866.942386:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654930880 00000100:00100000:3.0:1776188866.942428:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8209 00000100:00100000:3.0:1776188866.942434:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000673b41e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654930880:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1776188866.942502:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000673b41e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654930880:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 67us (448us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188866.942522:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8209 00000100:00100000:1.0:1776188866.942723:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57063 00000100:00100000:1.0:1776188866.942729:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000063864031 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654930816:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:1.0:1776188866.942754:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776188866.942783:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000063864031 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654930816:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 54us (818us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188866.942789:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57063 00010000:02020000:1.0:1776188867.164700:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776188867.260835:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005bd91e44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664246976:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776188867.260901:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188867.261373:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664246976 00000100:00100000:0.0:1776188867.261424:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6388 00000100:00100000:0.0:1776188867.261430:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007a0ce2a1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664246976:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776188867.261459:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776188867.261506:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007a0ce2a1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664246976:12345-0@lo:13:osp-pre-1-0.0 Request processed in 74us (604us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188867.261514:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6388 00000100:00100000:0.0:1776188867.261531:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6389 00000100:00100000:0.0:1776188867.261534:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009c73aa7b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664247040:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776188867.261551:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776188867.261578:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009c73aa7b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664247040:12345-0@lo:13:osp-pre-0-0.0 Request processed in 42us (139us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188867.261584:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6389 00000100:00100000:3.0:1776188867.261641:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000040b53b34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664247040:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776188867.261910:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005bd91e44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664246976:0@lo:13:osp-pre-1-0.0 00010000:02020000:1.0:1776188867.423161:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188867.676297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188867.932389:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188868.188304:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:1.0:1776188868.284484:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776188868.284491:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1776188868.284493:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b64538ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664247232:0@lo:400:kworker/u8:1.0 00000100:00080000:1.0:1776188868.284495:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776188868.284503:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9010 00000100:00080000:1.0:1776188868.284507:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1776188868.284511:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000a0e79be pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664247104:0@lo:400:kworker/u8:1.0 00000100:00080000:1.0:1776188868.284513:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776188868.284537:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000064636c08 x1862459664247296/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776188868.284566:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1776188868.284567:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188868.284571:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776188868.284581:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776188868.284583:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776188868.284587:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9010 00000100:00080000:1.0:1776188868.284589:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776188868.284593:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776188868.284605:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776188868.284605:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776188868.284606:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f1c366f3 x1862459664247424/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776188868.284617:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c0bb1b99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664247360:0@lo:400:kworker/u8:1.0 00000100:00080000:1.0:1776188868.284617:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1776188868.284618:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000205cde5a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664247168:0@lo:13:osp-pre-1-1.0 00000100:00080000:1.0:1776188868.284624:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776188868.284627:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776188868.284630:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9010 00000100:00080000:1.0:1776188868.284632:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776188868.284637:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776188868.284652:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006306c888 x1862459664247488/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776188868.284656:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188868.284670:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188868.284695:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664247104 00000100:00100000:3.0:1776188868.284703:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664247168 00000100:00100000:3.0:1776188868.284741:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6390 00000100:00100000:0.0:1776188868.284742:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1253 00000100:00100000:3.0:1776188868.284748:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005bd91e44 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664247168:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776188868.284752:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000089048673 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664247104:12345-0@lo:400:kworker/u8:1.0 00002000:00000024:3.0:1776188868.284770:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776188868.284800:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000089048673 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664247104:12345-0@lo:400:kworker/u8:1.0 Request processed in 50us (232us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188868.284810:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1253 00000100:00100000:3.0:1776188868.284813:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005bd91e44 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664247168:12345-0@lo:13:osp-pre-1-1.0 Request processed in 64us (143us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188868.284821:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6390 00000100:00100000:1.0:1776188868.284829:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000266f5a0c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664247744:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188868.284860:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000004e60a0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664247616:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188868.284867:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188868.284886:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776188868.284890:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b09edf48 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664247680:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188868.284896:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188868.284926:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776188868.284931:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cedf4ce4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664247552:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188868.284955:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188868.285010:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664247744 00000100:00100000:3.0:1776188868.285037:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664247616 00000100:00100000:0.0:1776188868.285045:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664247232 00000100:00100000:3.0:1776188868.285054:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664247552 00000100:00100000:0.0:1776188868.285079:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664247360 00000100:00100000:3.0:1776188868.285081:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8210 00000100:00100000:3.0:1776188868.285086:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fdab0c5c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664247744:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188868.285100:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664247296 00000100:00100000:0.0:1776188868.285116:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9549 00000100:00100000:0.0:1776188868.285120:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000072b3348c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664247232:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188868.286011:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8211 00000100:00100000:1.0:1776188868.286017:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003b5d789a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664247616:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188868.286066:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003b5d789a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664247616:12345-0@lo:400:kworker/u8:1.0 Request processed in 48us (1173us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188868.286072:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8211 00000100:00100000:1.0:1776188868.286089:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8212 00000100:00100000:1.0:1776188868.286093:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000013c0eaff pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664247552:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188868.286118:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000013c0eaff pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664247552:12345-0@lo:400:kworker/u8:1.0 Request processed in 25us (1165us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188868.286124:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8212 00000100:00100000:1.0:1776188868.286142:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9550 00000100:00100000:1.0:1776188868.286146:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003d6663c8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664247360:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188868.286198:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003d6663c8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664247360:12345-0@lo:400:kworker/u8:1.0 Request processed in 51us (1546us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188868.286204:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9550 00000100:00100000:1.0:1776188868.286218:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57064 00000100:00100000:1.0:1776188868.286222:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b94e9889 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664247296:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776188868.286234:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776188868.286260:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b94e9889 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664247296:12345-0@lo:38:kworker/u8:1.0 Request processed in 37us (1273us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188868.286274:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57064 00000100:00100000:1.0:1776188868.286500:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000264516d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664247808:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776188868.286529:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188868.286577:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000205cde5a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664247168:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776188868.286594:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000a0e79be pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664247104:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188868.286826:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664247424 00000100:00100000:1.0:1776188868.286861:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57065 00000100:00100000:1.0:1776188868.286867:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b6487cd9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664247424:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776188868.286876:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776188868.286938:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b6487cd9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664247424:12345-0@lo:38:kworker/u8:1.0 Request processed in 70us (298us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188868.286946:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57065 00000100:00100000:1.0:1776188868.287004:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c0bb1b99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664247360:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188868.287026:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fdab0c5c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664247744:12345-0@lo:400:kworker/u8:1.0 Request processed in 1936us (2156us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188868.287045:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8210 00000100:00100000:1.0:1776188868.287075:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cedf4ce4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664247552:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188868.287080:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664247808 00000100:00100000:1.0:1776188868.287091:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000004e60a0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664247616:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188868.287473:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000072b3348c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664247232:12345-0@lo:400:kworker/u8:1.0 Request processed in 2351us (2908us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188868.287481:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9549 00000100:00100000:1.0:1776188868.287552:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b64538ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664247232:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188868.288312:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6391 00000100:00100000:3.0:1776188868.288317:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d9b9c94e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664247808:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776188868.288337:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776188868.288389:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d9b9c94e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664247808:12345-0@lo:13:osp-pre-0-1.0 Request processed in 72us (1861us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188868.288395:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6391 00000100:00100000:0.0:1776188868.288599:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000264516d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664247808:0@lo:13:osp-pre-0-1.0 00010000:02020000:1.0:1776188868.444483:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188868.700318:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188868.956530:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188869.212306:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188869.468304:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776188869.498131:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188869.498262:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654930944 00000100:00100000:0.0:1776188869.498332:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57066 00000100:00100000:0.0:1776188869.498344:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000264516d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654930944:12345-192.168.203.25@tcp:17:kworker/3:2.0 00000100:00100000:0.0:1776188869.498429:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000264516d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654930944:12345-192.168.203.25@tcp:17:kworker/3:2.0 Request processed in 85us (306us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188869.498437:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57066 00010000:02020000:0.0:1776188869.724250:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188869.980460:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188870.236692:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188870.492309:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188870.748344:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188871.004305:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188871.260435:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188871.516303:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188871.772466:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188872.028618:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776188872.062512:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188872.062618:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188872.062676:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188872.062777:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776188872.062838:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654931200 00000100:00100000:1.0:1776188872.062888:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8213 00000100:00100000:1.0:1776188872.062895:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000229fc20a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2278:x1862459654931200:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776188872.062982:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000229fc20a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2278:x1862459654931200:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 77us (297us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188872.062990:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8213 00000100:00100000:0.0:1776188872.065589:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654931136 00000100:00100000:0.0:1776188872.065641:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654931072 00000100:00100000:0.0:1776188872.065678:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9552 00000100:00100000:0.0:1776188872.065683:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005c66c1fe pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654931072:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776188872.065796:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005c66c1fe pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654931072:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 112us (3019us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188872.069600:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9552 00010000:02020000:0.0:1776188872.284403:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188872.380691:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664247872 00000100:00100000:0.0:1776188872.380873:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6392 00000100:00100000:0.0:1776188872.380879:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006f76f00e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664247872:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776188872.380911:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776188872.380961:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006f76f00e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664247872:12345-0@lo:13:osp-pre-0-0.0 Request processed in 81us (367us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188872.380968:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6392 00000100:00100000:0.0:1776188872.381146:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000266f5a0c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664247936:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776188872.381195:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188872.381270:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664247936 00000100:00100000:1.0:1776188872.381303:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6393 00000100:00100000:1.0:1776188872.381306:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000013bfaae8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664247936:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776188872.381322:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776188872.381360:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000013bfaae8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664247936:12345-0@lo:13:osp-pre-1-0.0 Request processed in 53us (166us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188872.381366:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6393 00000100:00100000:0.0:1776188872.381608:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000266f5a0c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664247936:0@lo:13:osp-pre-1-0.0 00010000:02020000:0.0:1776188872.541687:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188872.799069:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188873.052484:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188873.308300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776188873.404556:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e0dcc408 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664248128:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188873.404638:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188873.404671:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776188873.404678:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a0c7825a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664248064:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776188873.404711:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188873.404748:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005e3f10b7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664248512:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188873.404782:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188873.404803:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776188873.404823:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f92d37ee pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664248576:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188873.404867:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188873.404907:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664248128 00000100:00100000:3.0:1776188873.404948:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664248192 00000100:00100000:3.0:1776188873.404963:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664248576 00000100:00100000:3.0:1776188873.404981:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9553 00000100:00100000:3.0:1776188873.404986:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e463b423 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664248128:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188873.405030:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e463b423 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664248128:12345-0@lo:400:kworker/u8:1.0 Request processed in 42us (395us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188873.405043:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9553 00000100:00100000:3.0:1776188873.405062:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9554 00000100:00100000:3.0:1776188873.405066:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000077c40bf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664248576:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188873.405093:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000077c40bf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664248576:12345-0@lo:400:kworker/u8:1.0 Request processed in 27us (236us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188873.405099:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9554 00000100:00100000:3.0:1776188873.405109:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57068 00000100:00100000:3.0:1776188873.405112:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006bbdf9dc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664248192:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776188873.405125:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188873.405150:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006bbdf9dc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664248192:12345-0@lo:38:kworker/u8:1.0 Request processed in 37us (332us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188873.405155:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57068 00000100:00100000:3.0:1776188873.405217:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188873.405562:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006306c888 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664248384:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776188873.405597:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1776188873.405626:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188873.405630:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188873.405635:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b09edf48 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664248192:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776188873.405661:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664248384 00000100:00100000:3.0:1776188873.405683:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9022 00000100:00100000:3.0:1776188873.405686:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a17ea634 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664248384:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776188873.405695:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188873.405725:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a17ea634 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664248384:12345-0@lo:38:kworker/u8:1.0 Request processed in 38us (128us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188873.405731:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9022 00000100:00100000:0.0:1776188873.406288:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000004e60a0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664248256:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188873.406418:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188873.406441:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008fea0698 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664248640:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188873.406496:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664248320 00000100:00100000:3.0:1776188873.406514:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664248256 00000100:00100000:3.0:1776188873.406531:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9555 00000100:00100000:3.0:1776188873.406535:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000887d8a72 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664248256:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188873.406550:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188873.406579:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000887d8a72 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664248256:12345-0@lo:400:kworker/u8:1.0 Request processed in 43us (162us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188873.406586:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9555 00000100:00100000:0.0:1776188873.406596:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000004e60a0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664248256:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188873.406604:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57069 00000100:00100000:3.0:1776188873.406609:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d9c93ad2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664248320:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776188873.406612:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00010000:02000000:3.0:1776188873.406620:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776188873.406624:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000026c12667 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664248000:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188873.406657:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d9c93ad2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664248320:12345-0@lo:38:kworker/u8:1.0 Request processed in 46us (1439us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188873.406664:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57069 00000100:00080000:3.0:1776188873.406692:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188873.406696:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188873.406703:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cedf4ce4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664248320:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776188873.406718:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188873.406720:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188873.406724:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006306c888 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664248384:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776188873.407021:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664248640 00000100:00100000:3.0:1776188873.407107:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8215 00000100:00100000:3.0:1776188873.407112:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000061dd8d04 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664248640:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188873.407408:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188873.409574:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d262f767 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664248448:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188873.409584:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000061dd8d04 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664248640:12345-0@lo:400:kworker/u8:1.0 Request processed in 2469us (3033us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188873.409600:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8215 00000100:00100000:3.0:1776188873.409664:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664248000 00000100:00100000:3.0:1776188873.409695:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1255 00000100:00100000:3.0:1776188873.409700:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c1af929a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664248000:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188873.409713:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188873.409744:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000046fee2b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664248704:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776188873.409749:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c1af929a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664248000:12345-0@lo:400:kworker/u8:1.0 Request processed in 45us (2339us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188873.409757:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1255 00000100:00100000:0.0:1776188873.409778:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188873.409872:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008fea0698 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664248640:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188873.409889:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000026c12667 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664248000:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188873.409897:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d262f767 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664248448:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188873.410045:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664248704 00000100:00100000:3.0:1776188873.410069:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6395 00000100:00100000:3.0:1776188873.410073:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003fb63a1c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664248704:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776188873.410085:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776188873.410114:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003fb63a1c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664248704:12345-0@lo:13:osp-pre-1-1.0 Request processed in 40us (339us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188873.410120:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6395 00000100:00100000:0.0:1776188873.410222:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000046fee2b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664248704:0@lo:13:osp-pre-1-1.0 00010000:02020000:0.0:1776188873.564486:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188873.820377:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188874.077438:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188874.332304:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188874.588287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188874.847523:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188875.100283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188875.356476:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188875.612337:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188875.868282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188876.124688:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188876.382233:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188876.636299:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188876.892307:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188877.148281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776188877.179050:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654931328 00000100:00100000:3.0:1776188877.179230:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654931264 00000100:00100000:1.0:1776188877.179265:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9556 00000100:00100000:1.0:1776188877.179274:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005e3f10b7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654931328:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776188877.179327:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1256 00000100:00100000:3.0:1776188877.179334:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e2219b01 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654931264:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776188877.179368:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005e3f10b7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654931328:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 94us (394us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188877.179371:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e2219b01 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654931264:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 35us (450us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188877.179386:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9556 00000100:00100000:3.0:1776188877.179391:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1256 00000100:00100000:1.0:1776188877.184706:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776188877.184754:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188877.186050:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654931456 00000100:00100000:0.0:1776188877.186141:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8217 00000100:00100000:0.0:1776188877.186145:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b64538ad pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2278:x1862459654931456:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776188877.186215:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b64538ad pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2278:x1862459654931456:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 68us (1461us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188877.186226:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8217 00010000:02020000:1.0:1776188877.404391:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776188877.501062:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f1c366f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664248832:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776188877.501107:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188877.501204:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664248768 00000100:00100000:0.0:1776188877.501286:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664248832 00000100:00100000:1.0:1776188877.503793:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6396 00000100:00100000:1.0:1776188877.503805:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f92d37ee pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664248768:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776188877.503845:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776188877.507897:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f1c366f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664248832:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776188877.508636:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f92d37ee pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664248768:12345-0@lo:13:osp-pre-0-0.0 Request processed in 4830us (7614us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188877.508843:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6396 00010000:02020000:1.0:1776188877.661390:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188877.916659:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188878.172295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188878.430224:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:3.0:1776188878.524918:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188878.524924:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188878.524928:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188878.524934:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9020 00000100:00080000:3.0:1776188878.524938:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00080000:3.0:1776188878.524944:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188878.524965:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000db2d4fd6 x1862459664249088/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776188878.524998:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188878.525002:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188878.525003:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188878.525007:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9020 00000100:00080000:3.0:1776188878.525011:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00080000:3.0:1776188878.525016:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188878.525027:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000078cf1bd6 x1862459664249216/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776188878.525036:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188878.525041:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188878.525043:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188878.525048:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9020 00000100:00080000:3.0:1776188878.525050:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 6s 00000100:00080000:3.0:1776188878.525054:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188878.525080:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003f45b220 x1862459664249280/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776188878.525139:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c494ab9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664249536:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188878.525207:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188878.525238:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000db2d4fd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664249088:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188878.525266:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188878.525384:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664249536 00000100:00100000:0.0:1776188878.525409:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8218 00000100:00100000:0.0:1776188878.525412:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f600e06c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664249536:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188878.525458:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f600e06c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664249536:12345-0@lo:400:kworker/u8:0.0 Request processed in 44us (253us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188878.525464:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8218 00000100:00100000:0.0:1776188878.525495:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c494ab9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664249536:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188878.537129:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000078cf1bd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664249216:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188878.537212:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188878.538325:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9558 00000100:00100000:0.0:1776188878.538331:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d262f767 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664249152:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188878.538394:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9559 00000100:00100000:1.0:1776188878.538401:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008fea0698 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664249472:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188878.544142:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d262f767 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664249152:12345-0@lo:400:kworker/u8:0.0 Request processed in 5803us (19038us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188878.544160:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9558 00000100:00100000:0.0:1776188878.544219:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664249216 00000100:00100000:0.0:1776188878.544250:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57071 00000100:00100000:0.0:1776188878.544255:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d7babd95 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664249088:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776188878.544271:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776188878.544624:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d7babd95 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664249088:12345-0@lo:38:kworker/u8:0.0 Request processed in 368us (19358us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188878.544632:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57071 00000100:00100000:0.0:1776188878.544699:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57072 00000100:00100000:0.0:1776188878.544703:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000099b0e8b4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664249216:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776188878.544712:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776188878.544814:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000099b0e8b4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664249216:12345-0@lo:38:kworker/u8:0.0 Request processed in 110us (7604us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188878.544820:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57072 00000100:00100000:0.0:1776188878.544950:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ea0541f6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664249152:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188878.545027:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e75ea3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664248960:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188878.545063:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188878.545096:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000db5ec222 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664249408:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188878.545120:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188878.545138:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1776188878.545144:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe5f85f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664249344:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188878.545163:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188878.545199:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b29992fd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664249600:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776188878.545226:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188878.545252:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664249408 00000100:00100000:0.0:1776188878.545276:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664249344 00000100:00100000:0.0:1776188878.545299:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8219 00000100:00100000:0.0:1776188878.545303:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000024c95b46 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664249408:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188878.545343:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000024c95b46 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664249408:12345-0@lo:400:kworker/u8:0.0 Request processed in 39us (222us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188878.545349:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8219 00000100:00100000:0.0:1776188878.545360:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8220 00000100:00100000:0.0:1776188878.545363:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009e6a2c02 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664249344:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188878.545468:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009e6a2c02 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664249344:12345-0@lo:400:kworker/u8:0.0 Request processed in 104us (305us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188878.545478:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8220 00000100:00100000:0.0:1776188878.545510:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe5f85f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664249344:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188878.545519:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000db5ec222 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664249408:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188878.545540:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664249600 00000100:00100000:0.0:1776188878.545566:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6399 00000100:00100000:0.0:1776188878.545569:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba2818a5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664249600:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776188878.545582:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776188878.545613:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba2818a5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664249600:12345-0@lo:13:osp-pre-0-1.0 Request processed in 42us (387us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188878.545619:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6399 00000100:00100000:0.0:1776188878.545961:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b29992fd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664249600:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776188878.546020:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002c842ea9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664248896:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776188878.546049:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664248960 00000100:00100000:0.0:1776188878.546083:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1257 00000100:00100000:0.0:1776188878.546087:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003e184549 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664248960:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188878.546123:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003e184549 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664248960:12345-0@lo:400:kworker/u8:0.0 Request processed in 35us (1059us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188878.546129:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1257 00000100:00100000:0.0:1776188878.546162:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e75ea3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664248960:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188878.547144:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009ac61361 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664249024:0@lo:400:kworker/u8:0.0 00000100:00080000:3.0:1776188878.547641:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188878.547648:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188878.547652:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000078cf1bd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664249216:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188878.547662:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003f45b220 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664249280:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188878.547705:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1776188878.547752:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188878.547756:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188878.547759:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000db2d4fd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664249088:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776188878.548168:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008fea0698 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664249472:12345-0@lo:400:kworker/u8:0.0 Request processed in 9765us (23025us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188878.548206:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9559 00000100:00080000:3.0:1776188878.548392:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188878.548397:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188878.548400:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003f45b220 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664249280:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776188878.549993:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000101b4f14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664249472:0@lo:400:kworker/u8:0.0 00010000:02020000:1.0:1776188878.684287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188878.941156:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188879.197353:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188879.454112:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188879.708771:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188879.964262:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188880.220293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188880.476311:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188880.732292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188880.988290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188881.244282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188881.500300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188881.756492:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188882.012406:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188882.268397:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776188882.300051:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776188882.300108:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188882.300250:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654931584 00000100:00100000:3.0:1776188882.300351:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57073 00000100:00100000:3.0:1776188882.300360:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000a0e79be pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654931648:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:3.0:1776188882.300382:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776188882.300452:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000a0e79be pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654931648:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 90us (341us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188882.300463:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57073 00000100:00100000:3.0:1776188882.308319:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188882.308379:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188882.308457:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654931520 00000100:00100000:0.0:1776188882.308534:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1258 00000100:00100000:0.0:1776188882.308540:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000089ddaa97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654931520:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776188882.308620:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000089ddaa97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654931520:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 80us (314us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188882.308627:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1258 00010000:02020000:1.0:1776188882.524314:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776188882.622343:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664249664 00000100:00100000:3.0:1776188882.622433:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6400 00000100:00100000:3.0:1776188882.622440:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000026c12667 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664249664:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776188882.622487:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776188882.622586:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000026c12667 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664249664:12345-0@lo:13:osp-pre-0-0.0 Request processed in 145us (314us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188882.622595:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6400 00000100:00100000:0.0:1776188882.623810:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f1c366f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664249728:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776188882.623859:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188882.625397:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664249728 00000100:00100000:1.0:1776188882.625469:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6401 00000100:00100000:1.0:1776188882.625475:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b29992fd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664249728:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776188882.625509:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776188882.625587:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b29992fd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664249728:12345-0@lo:13:osp-pre-1-0.0 Request processed in 111us (1729us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188882.625594:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6401 00000100:00100000:0.0:1776188882.626150:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f1c366f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664249728:0@lo:13:osp-pre-1-0.0 00010000:02020000:1.0:1776188882.780312:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188883.036417:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188883.292299:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188883.548292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188883.644479:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000acf34144 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664249856:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776188883.644501:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000db2d4fd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664249792:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188883.644553:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188883.644564:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188883.644608:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664249856 00000100:00100000:1.0:1776188883.644614:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664249792 00000100:00100000:0.0:1776188883.644649:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6402 00000100:00100000:0.0:1776188883.644658:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000affce7e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664249856:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776188883.644658:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1259 00000100:00100000:1.0:1776188883.644666:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006e7c9c41 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664249792:12345-0@lo:400:kworker/u8:0.0 00002000:00000024:0.0:1776188883.644681:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776188883.644721:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006e7c9c41 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664249792:12345-0@lo:400:kworker/u8:0.0 Request processed in 55us (158us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188883.644730:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1259 00000100:00100000:0.0:1776188883.644738:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000affce7e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664249856:12345-0@lo:13:osp-pre-0-1.0 Request processed in 82us (185us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188883.644746:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6402 00000100:00100000:1.0:1776188883.644786:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000db2d4fd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664249792:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188883.644790:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000acf34144 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664249856:0@lo:13:osp-pre-0-1.0 00000100:00080000:3.0:1776188883.644950:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188883.644954:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188883.644966:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188883.644970:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9025 00000100:00080000:3.0:1776188883.644973:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00080000:3.0:1776188883.644977:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188883.644995:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e5080bed x1862459664249984/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776188883.645028:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188883.645031:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188883.645033:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188883.645036:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9025 00000100:00080000:3.0:1776188883.645037:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:3.0:1776188883.645040:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188883.645049:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000da51a899 x1862459664250112/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776188883.645057:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188883.645060:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188883.645061:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188883.645064:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9025 00000100:00080000:3.0:1776188883.645065:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 11s 00000100:00080000:3.0:1776188883.645068:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188883.645077:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000db2d4fd6 x1862459664250176/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776188883.645096:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776188883.645108:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009ac61361 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664250304:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188883.645137:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188883.645164:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000078cf1bd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664249920:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188883.645208:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188883.645221:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776188883.645226:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e75ea3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664250368:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188883.645243:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188883.647591:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664250496 00000100:00100000:1.0:1776188883.647654:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6403 00000100:00100000:1.0:1776188883.647659:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004ac5ba05 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664250496:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776188883.647679:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776188883.647746:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004ac5ba05 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664250496:12345-0@lo:13:osp-pre-1-1.0 Request processed in 86us (1473us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188883.647753:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6403 00000100:00100000:1.0:1776188883.647778:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e5080bed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664249984:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776188883.647811:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188883.647831:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000da51a899 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664250112:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776188883.647854:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188883.647869:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000db2d4fd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664250176:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776188883.647891:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188883.647922:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664250176 00000100:00100000:1.0:1776188883.647942:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9024 00000100:00100000:1.0:1776188883.647946:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000021503dd8 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664250176:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776188883.647971:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776188883.648007:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000021503dd8 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664250176:12345-0@lo:38:kworker/u8:0.0 Request processed in 60us (115us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188883.648013:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9024 00000100:00100000:1.0:1776188883.648044:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664249984 00000100:00100000:1.0:1776188883.648062:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664250112 00000100:00100000:1.0:1776188883.648071:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57074 00000100:00100000:1.0:1776188883.648075:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f1c366f3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664249984:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776188883.648082:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776188883.648108:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f1c366f3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664249984:12345-0@lo:38:kworker/u8:0.0 Request processed in 33us (297us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188883.648114:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57074 00000100:00100000:1.0:1776188883.648123:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57075 00000100:00100000:1.0:1776188883.648126:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004d99e80b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664250112:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776188883.648132:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776188883.648152:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004d99e80b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664250112:12345-0@lo:38:kworker/u8:0.0 Request processed in 25us (299us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188883.648157:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57075 00000100:00080000:1.0:1776188883.648224:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776188883.648228:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776188883.648232:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e5080bed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664249984:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776188883.648239:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776188883.648242:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776188883.648244:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000da51a899 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664250112:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776188883.648251:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776188883.648253:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776188883.648270:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000db2d4fd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664250176:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188883.649050:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f34aeb18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664250496:0@lo:13:osp-pre-1-1.0 00010000:02020000:1.0:1776188883.804292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188884.060296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188884.316389:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188884.573252:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188884.828698:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188885.084370:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188885.342958:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188885.597083:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188885.852343:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188886.109142:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188886.365594:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188886.620278:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188886.876316:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188887.132836:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188887.388568:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776188887.423339:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188887.423393:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188887.423431:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188887.423478:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776188887.425102:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654931776 00000100:00100000:1.0:1776188887.425156:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1260 00000100:00100000:1.0:1776188887.425161:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d3d96460 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654931776:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1776188887.425236:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d3d96460 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654931776:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 75us (1906us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188887.425243:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1260 00000100:00100000:1.0:1776188887.425276:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654931968 00000100:00100000:1.0:1776188887.425308:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8225 00000100:00100000:1.0:1776188887.425312:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ea0541f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2280:x1862459654931968:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1776188887.425335:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ea0541f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2280:x1862459654931968:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 22us (1905us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188887.425340:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8225 00010000:02020000:1.0:1776188887.644284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188887.740517:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776188887.740540:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000914d0a5f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664250560:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776188887.740610:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188887.740659:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664250560 00000100:00100000:0.0:1776188887.740701:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6404 00000100:00100000:0.0:1776188887.740707:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fdda22fa pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664250560:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776188887.740734:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776188887.740774:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fdda22fa pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664250560:12345-0@lo:13:osp-pre-0-0.0 Request processed in 66us (164us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188887.740781:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6404 00000100:00100000:0.0:1776188887.740824:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000914d0a5f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664250560:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776188887.741094:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776188887.741104:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000071348c99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664250624:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776188887.741135:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188887.741326:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664250624 00000100:00100000:1.0:1776188887.741358:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6405 00000100:00100000:1.0:1776188887.741362:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0c7825a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664250624:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776188887.741381:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776188887.741422:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0c7825a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664250624:12345-0@lo:13:osp-pre-1-0.0 Request processed in 59us (287us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188887.741428:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6405 00000100:00100000:3.0:1776188887.741626:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000071348c99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664250624:0@lo:13:osp-pre-1-0.0 00010000:02020000:1.0:1776188887.900323:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188888.156573:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188888.413913:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188888.668519:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:3.0:1776188888.764364:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188888.764371:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188888.764374:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188888.764379:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9030 00000100:00080000:3.0:1776188888.764383:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1776188888.764387:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188888.764404:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e5080bed x1862459664250880/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776188888.764428:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188888.764430:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188888.764431:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188888.764434:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9030 00000100:00080000:3.0:1776188888.764436:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1776188888.764439:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188888.764447:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009ac61361 x1862459664251008/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776188888.764454:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188888.764456:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188888.764458:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188888.764460:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9030 00000100:00080000:3.0:1776188888.764461:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1776188888.764464:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188888.764472:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000e75ea3b x1862459664251072/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776188888.764656:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000db2d4fd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664250752:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188888.764689:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188888.764703:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c89da359 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664251200:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188888.764720:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188888.764732:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:1.0:1776188888.764736:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f34aeb18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664250688:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776188888.764766:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188888.764783:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000db5ec222 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664251136:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188888.764799:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188888.764842:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664250816 00000100:00100000:3.0:1776188888.764880:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664251264 00000100:00100000:3.0:1776188888.764895:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664250944 00000100:00100000:3.0:1776188888.764910:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9565 00000100:00100000:3.0:1776188888.764913:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000266e2268 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664250816:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188888.764944:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000266e2268 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664250816:12345-0@lo:400:kworker/u8:1.0 Request processed in 30us (481us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188888.764949:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9565 00000100:00100000:3.0:1776188888.764961:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9566 00000100:00100000:3.0:1776188888.764964:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ee8b6459 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664251264:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188888.764986:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ee8b6459 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664251264:12345-0@lo:400:kworker/u8:1.0 Request processed in 21us (455us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188888.764991:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9566 00000100:00100000:3.0:1776188888.765000:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9567 00000100:00100000:3.0:1776188888.765004:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dabf4d62 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664250944:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188888.765026:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dabf4d62 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664250944:12345-0@lo:400:kworker/u8:1.0 Request processed in 21us (450us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188888.765032:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9567 00000100:00100000:3.0:1776188888.765061:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664250752 00000100:00100000:3.0:1776188888.765081:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1261 00000100:00100000:3.0:1776188888.765085:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000027854250 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664250752:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188888.765109:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000027854250 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664250752:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (420us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188888.765114:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1261 00000100:00100000:3.0:1776188888.765146:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e5080bed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664250880:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776188888.765168:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188888.765208:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009ac61361 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664251008:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776188888.765229:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188888.765241:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e75ea3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664251072:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776188888.765262:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188888.765573:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000db5ec222 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664251136:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188888.765579:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cedf4ce4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664251392:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188888.765600:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188888.765618:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000db2d4fd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664250752:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188888.765627:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c89da359 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664251200:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188888.765652:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664250688 00000100:00100000:1.0:1776188888.765675:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6406 00000100:00100000:1.0:1776188888.765680:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d2d0e609 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664250688:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776188888.765694:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776188888.765722:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d2d0e609 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3362:x1862459664250688:12345-0@lo:13:osp-pre-1-1.0 Request processed in 43us (957us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188888.765728:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6406 00000100:00100000:1.0:1776188888.765760:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f34aeb18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664250688:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776188888.765768:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cedf4ce4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664251392:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188888.766143:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000037fc1275 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664251456:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776188888.766188:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188888.766413:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664251456 00000100:00100000:1.0:1776188888.766444:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6407 00000100:00100000:1.0:1776188888.766447:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000914d0a5f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664251456:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776188888.766458:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776188888.766495:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000914d0a5f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664251456:12345-0@lo:13:osp-pre-0-1.0 Request processed in 47us (308us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188888.766501:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6407 00000100:00080000:3.0:1776188888.766608:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188888.766612:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188888.766616:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e5080bed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664250880:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776188888.766628:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188888.766631:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188888.766633:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009ac61361 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664251008:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776188888.766641:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188888.766643:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188888.766645:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e75ea3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664251072:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776188888.766724:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000037fc1275 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664251456:0@lo:13:osp-pre-0-1.0 00010000:02020000:1.0:1776188888.924309:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188889.180543:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188889.438217:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188889.692347:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188889.948401:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188890.204286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188890.460285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188890.716565:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188890.972297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188891.228285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188891.484294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188891.740340:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188891.996347:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188892.252549:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188892.508324:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776188892.539138:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188892.539291:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188892.540954:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654932096 00000100:00100000:0.0:1776188892.541022:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9568 00000100:00100000:0.0:1776188892.541028:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cedf4ce4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654932096:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776188892.541121:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cedf4ce4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654932096:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 82us (1821us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188892.541128:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9568 00000100:00100000:1.0:1776188892.542063:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776188892.542148:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188892.542252:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654932224 00000100:00100000:0.0:1776188892.542299:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8230 00000100:00100000:0.0:1776188892.542304:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000025a2ff5a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2278:x1862459654932224:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776188892.542353:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000025a2ff5a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2278:x1862459654932224:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 49us (206us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188892.542360:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8230 00000100:00100000:3.0:1776188892.542512:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654932160 00000100:00100000:3.0:1776188892.542545:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57079 00000100:00100000:3.0:1776188892.542550:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000797ba66f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654932160:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:3.0:1776188892.542565:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776188892.542607:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000797ba66f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654932160:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 56us (548us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188892.542614:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57079 00010000:02020000:1.0:1776188892.764974:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776188892.860425:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776188892.860449:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001b6b6588 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664251520:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776188892.860538:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188892.860818:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664251520 00000100:00100000:1.0:1776188892.860884:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6408 00000100:00100000:1.0:1776188892.860890:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e13527ae pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664251520:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776188892.860919:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776188892.861088:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e13527ae pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664251520:12345-0@lo:13:osp-pre-0-0.0 Request processed in 197us (551us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188892.861097:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6408 00000100:00100000:0.0:1776188892.861206:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028933edf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664251584:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776188892.861236:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188892.861277:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001b6b6588 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664251520:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776188892.861376:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664251584 00000100:00100000:1.0:1776188892.861437:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6409 00000100:00100000:1.0:1776188892.861440:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e139d197 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664251584:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776188892.861458:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776188892.861495:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e139d197 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664251584:12345-0@lo:13:osp-pre-1-0.0 Request processed in 54us (259us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188892.861501:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6409 00000100:00100000:0.0:1776188892.861643:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028933edf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664251584:0@lo:13:osp-pre-1-0.0 00010000:02020000:1.0:1776188893.020303:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188893.276471:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188893.532321:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188893.788285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776188893.884445:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001b6b6588 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664251648:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776188893.884515:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188893.884553:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776188893.884559:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007444cfc6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664251712:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776188893.884581:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188893.884657:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664251712 00000100:00080000:3.0:1776188893.884812:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188893.884817:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188893.884820:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188893.884825:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9035 00000100:00080000:3.0:1776188893.884829:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776188893.884832:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188893.884847:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007390d733 x1862459664251904/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776188893.884871:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188893.884874:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188893.884875:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188893.884878:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9035 00000100:00080000:3.0:1776188893.884880:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776188893.884883:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188893.884891:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e0409ecf x1862459664252032/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776188893.884898:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188893.884900:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188893.884902:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188893.884904:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9035 00000100:00080000:3.0:1776188893.884906:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776188893.884908:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188893.884917:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d3fa5d0f x1862459664252096/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776188893.884953:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008870952c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664251776:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188893.884986:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188893.885003:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000699c9233 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664252224:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188893.885025:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188893.885042:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1776188893.885047:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000080b4f511 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664251840:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188893.885071:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188893.885087:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002046cfde pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664252288:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188893.885107:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188893.885127:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007390d733 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664251904:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188893.885149:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188893.885162:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e0409ecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664252032:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188893.885203:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188893.885752:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6411 00000100:00100000:0.0:1776188893.885758:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c68f859e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664251712:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776188893.885777:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776188893.885891:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c68f859e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664251712:12345-0@lo:13:osp-pre-1-1.0 Request processed in 133us (1310us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188893.885909:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6411 00000100:00100000:0.0:1776188893.885943:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664251840 00000100:00100000:0.0:1776188893.885975:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664252288 00000100:00100000:3.0:1776188893.885996:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d3fa5d0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664252096:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188893.886037:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188893.886073:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8a51369 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664252160:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188893.886095:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188893.886110:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776188893.886114:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc4a257b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664251968:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188893.886130:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188893.886407:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9569 00000100:00100000:0.0:1776188893.886413:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cfec04c5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664251840:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188893.886460:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cfec04c5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664251840:12345-0@lo:400:kworker/u8:0.0 Request processed in 47us (1389us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188893.886467:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9569 00000100:00100000:3.0:1776188893.886526:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664251776 00000100:00100000:3.0:1776188893.886565:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1263 00000100:00100000:3.0:1776188893.886573:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b64e4d66 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664251776:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188893.886606:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b64e4d66 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664251776:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (1619us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188893.886612:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1263 00000100:00100000:3.0:1776188893.886640:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9571 00000100:00100000:3.0:1776188893.886643:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a02f1c47 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664251968:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188893.886669:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a02f1c47 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664251968:12345-0@lo:400:kworker/u8:0.0 Request processed in 25us (538us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188893.886674:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9571 00000100:00100000:3.0:1776188893.886685:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57080 00000100:00100000:3.0:1776188893.886688:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001c8daed5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664251904:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776188893.886699:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188893.886722:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001c8daed5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664251904:12345-0@lo:38:kworker/u8:0.0 Request processed in 33us (1574us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188893.886727:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57080 00000100:00100000:3.0:1776188893.886747:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57081 00000100:00100000:3.0:1776188893.886750:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000018d4eed1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664252032:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776188893.886756:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188893.886775:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000018d4eed1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664252032:12345-0@lo:38:kworker/u8:0.0 Request processed in 24us (1573us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188893.886779:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57081 00000100:00100000:3.0:1776188893.886794:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664252096 00000100:00100000:3.0:1776188893.886809:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9026 00000100:00100000:3.0:1776188893.886812:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000761163f9 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664252096:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776188893.886818:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188893.886837:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000761163f9 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664252096:12345-0@lo:38:kworker/u8:0.0 Request processed in 24us (811us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188893.886842:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9026 00000100:00100000:3.0:1776188893.886876:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664252224 00000100:00100000:3.0:1776188893.886899:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664252160 00000100:00100000:3.0:1776188893.886918:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8231 00000100:00100000:3.0:1776188893.886923:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000528f790 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664252224:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188893.886937:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007444cfc6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664251712:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776188893.886963:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000528f790 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664252224:12345-0@lo:400:kworker/u8:0.0 Request processed in 38us (1936us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188893.886967:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001b6b6588 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664251648:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776188893.886972:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8231 00000100:00100000:3.0:1776188893.886993:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8232 00000100:00100000:3.0:1776188893.886999:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007a388dd2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664252160:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188893.887022:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007a388dd2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664252160:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (927us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188893.887040:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8232 00000100:00100000:1.0:1776188893.887054:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000080b4f511 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664251840:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188893.887077:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002046cfde pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664252288:0@lo:400:kworker/u8:0.0 00000100:00080000:3.0:1776188893.887081:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188893.887087:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188893.887093:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007390d733 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664251904:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776188893.887094:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008870952c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664251776:0@lo:400:kworker/u8:0.0 00000100:00080000:3.0:1776188893.887110:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188893.887115:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776188893.887116:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000699c9233 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664252224:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188893.887121:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e0409ecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664252032:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776188893.887137:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188893.887140:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188893.887143:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d3fa5d0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664252096:0@lo:38:kworker/u8:0.0 00010000:02020000:1.0:1776188894.044287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188894.300291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188894.557026:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188894.813208:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188895.068840:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188895.324290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188895.580869:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188895.836279:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188896.092289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188896.348350:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188896.604283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188896.862020:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188897.116280:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188897.372293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188897.629948:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776188897.667686:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188897.667733:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188897.667779:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188897.667818:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654932480 00000100:00100000:3.0:1776188897.667865:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8233 00000100:00100000:3.0:1776188897.667871:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007390d733 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2279:x1862459654932480:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1776188897.667940:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007390d733 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2279:x1862459654932480:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 67us (160us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188897.667947:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8233 00000100:00100000:1.0:1776188897.668046:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654932352 00000100:00100000:1.0:1776188897.668084:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654932416 00000100:00100000:1.0:1776188897.668115:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9572 00000100:00100000:1.0:1776188897.668119:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d3fa5d0f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654932352:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1776188897.668163:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d3fa5d0f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654932352:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 43us (487us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188897.668170:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9572 00000100:00100000:1.0:1776188897.668210:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57082 00000100:00100000:1.0:1776188897.668214:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e0409ecf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654932416:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:1.0:1776188897.668228:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776188897.668251:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e0409ecf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654932416:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 36us (520us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188897.668256:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57082 00000100:00100000:1.0:1776188897.674835:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:02020000:1.0:1776188897.884449:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776188897.980954:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664252352 00000100:00100000:1.0:1776188897.981005:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664252416 00000100:00100000:1.0:1776188897.981139:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6412 00000100:00100000:1.0:1776188897.981146:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000094a1a2a8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664252352:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776188897.981152:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6413 00000100:00100000:3.0:1776188897.981159:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000516ddb7f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664252416:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776188897.981199:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00002000:00000024:3.0:1776188897.981209:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776188897.981254:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000094a1a2a8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664252352:12345-0@lo:13:osp-pre-0-0.0 Request processed in 108us (448us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188897.981257:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000516ddb7f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664252416:12345-0@lo:13:osp-pre-1-0.0 Request processed in 97us (372us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188897.981264:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6413 00000100:00100000:1.0:1776188897.981274:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6412 00000100:00100000:1.0:1776188897.981321:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008d1771b5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664252416:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776188897.981333:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094ac8433 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664252352:0@lo:13:osp-pre-0-0.0 00010000:02020000:1.0:1776188898.140293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188898.396364:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188898.652278:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188898.909187:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188899.004384:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776188899.004546:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776188899.004579:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000b321ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664252544:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188899.004765:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188899.004809:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664252544 00000100:00100000:1.0:1776188899.004849:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9573 00000100:00100000:1.0:1776188899.004856:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f85662d8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664252544:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188899.004906:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f85662d8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664252544:12345-0@lo:400:kworker/u8:0.0 Request processed in 50us (143us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188899.004913:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9573 00000100:00100000:1.0:1776188899.004937:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664252608 00000100:00100000:1.0:1776188899.004960:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664252672 00000100:00100000:1.0:1776188899.004981:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6414 00000100:00100000:1.0:1776188899.004986:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007d51703e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664252608:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776188899.005001:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776188899.005032:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000085a66d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664252480:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188899.005165:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007d51703e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664252608:12345-0@lo:13:osp-pre-0-1.0 Request processed in 178us (395us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188899.005201:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6414 00000100:00100000:1.0:1776188899.005241:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008d1771b5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664252608:0@lo:13:osp-pre-0-1.0 00000100:00080000:3.0:1776188899.005276:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188899.005282:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188899.005285:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188899.005289:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9040 00000100:00080000:3.0:1776188899.005291:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776188899.005294:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188899.005314:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bc501d10 x1862459664252736/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776188899.005336:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188899.005339:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188899.005340:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188899.005342:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9040 00000100:00080000:3.0:1776188899.005344:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776188899.005347:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188899.005355:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b9d589b6 x1862459664252864/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776188899.005363:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188899.005365:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188899.005366:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188899.005369:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9040 00000100:00080000:3.0:1776188899.005370:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776188899.005373:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188899.005382:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a7368321 x1862459664252928/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776188899.005419:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000b321ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664252544:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188899.005437:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002ea4a019 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664252800:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188899.005464:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188899.005481:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776188899.005486:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006ad1d61a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664252992:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188899.005510:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188899.005536:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000053d4c624 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664253056:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188899.005549:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bc501d10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664252736:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188899.005582:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188899.009466:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188899.009520:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776188899.009529:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8a51369 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664253120:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188899.009572:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188899.010003:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664252992 00000100:00100000:1.0:1776188899.010055:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664253056 00000100:00100000:1.0:1776188899.010078:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8234 00000100:00100000:1.0:1776188899.010082:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d6ea221c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664252992:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188899.010138:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d6ea221c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664252992:12345-0@lo:400:kworker/u8:0.0 Request processed in 55us (4629us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188899.010144:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8234 00000100:00100000:1.0:1776188899.010161:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8235 00000100:00100000:1.0:1776188899.010164:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000def45ccb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664253056:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188899.010221:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000def45ccb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664253056:12345-0@lo:400:kworker/u8:0.0 Request processed in 56us (763us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188899.010230:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8235 00000100:00100000:1.0:1776188899.010255:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664252800 00000100:00100000:1.0:1776188899.010438:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664252736 00000100:00100000:1.0:1776188899.010453:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664253120 00000100:00100000:1.0:1776188899.010469:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9574 00000100:00100000:1.0:1776188899.010472:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008d1771b5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664252800:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188899.010506:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008d1771b5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664252800:12345-0@lo:400:kworker/u8:0.0 Request processed in 33us (5044us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188899.010511:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9574 00000100:00100000:1.0:1776188899.010522:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9575 00000100:00100000:1.0:1776188899.010526:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009cace85d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664253120:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188899.010549:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009cace85d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664253120:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (977us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188899.010567:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9575 00000100:00100000:1.0:1776188899.010578:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57083 00000100:00100000:1.0:1776188899.010582:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dc4a257b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664252736:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776188899.010598:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776188899.010599:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188899.010657:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664252480 00000100:00100000:0.0:1776188899.010686:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1265 00000100:00100000:0.0:1776188899.010691:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002e45a4e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664252480:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188899.010718:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dc4a257b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664252736:12345-0@lo:38:kworker/u8:0.0 Request processed in 134us (5136us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188899.010726:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57083 00000100:00100000:0.0:1776188899.010739:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002e45a4e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664252480:12345-0@lo:400:kworker/u8:0.0 Request processed in 46us (142us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188899.010747:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1265 00000100:00100000:0.0:1776188899.010796:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000085a66d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664252480:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188899.010845:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006ad1d61a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664252992:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188899.010856:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002ea4a019 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664252800:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188899.011411:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8a51369 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664253120:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188899.011427:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000053d4c624 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664253056:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188899.011442:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b181f293 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664252672:0@lo:13:osp-pre-1-1.0 00000100:00080000:3.0:1776188899.012005:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188899.012010:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188899.012015:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bc501d10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664252736:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188899.012024:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b9d589b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664252864:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188899.012068:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188899.012236:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a7368321 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664252928:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188899.012261:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776188899.012698:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776188899.012703:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776188899.012706:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a7368321 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664252928:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776188899.014742:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664252864 00000100:00100000:0.0:1776188899.014800:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57084 00000100:00100000:0.0:1776188899.014806:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000699c9233 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664252864:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776188899.014822:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776188899.014889:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000699c9233 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664252864:12345-0@lo:38:kworker/u8:0.0 Request processed in 83us (2821us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188899.014896:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57084 00000100:00080000:1.0:1776188899.015758:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776188899.015765:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776188899.015771:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b9d589b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664252864:0@lo:38:kworker/u8:0.0 00010000:02020000:1.0:1776188899.166820:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188899.420374:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188899.676283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188899.932668:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188900.188279:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188900.444296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188900.700287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188900.956283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188901.212332:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188901.469075:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188901.726001:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188901.982385:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188902.236345:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188902.492256:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188902.748284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776188902.781940:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776188902.781986:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776188902.782035:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188902.782209:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654932608 00000100:00100000:3.0:1776188902.782259:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9576 00000100:00100000:3.0:1776188902.782265:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000012b5e917 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654932608:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1776188902.782341:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000012b5e917 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654932608:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 74us (305us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188902.782350:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9576 00000100:00100000:0.0:1776188902.782360:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654932544 00000100:00100000:0.0:1776188902.782397:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1266 00000100:00100000:0.0:1776188902.782403:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002ea0417b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654932544:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776188902.782435:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002ea0417b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654932544:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 32us (502us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188902.782442:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1266 00000100:00100000:3.0:1776188902.782469:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188902.782496:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654932672 00000100:00100000:0.0:1776188902.782522:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57085 00000100:00100000:0.0:1776188902.782528:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bc501d10 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654932672:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:0.0:1776188902.782556:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776188902.782592:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bc501d10 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654932672:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 63us (125us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188902.782600:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57085 00010000:02020000:0.0:1776188903.006050:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188903.102299:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664253184 00000100:00100000:0.0:1776188903.102337:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664253248 00000100:00100000:0.0:1776188903.102355:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6416 00000100:00100000:0.0:1776188903.102359:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b7c9d478 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664253184:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776188903.102382:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776188903.102401:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6417 00000100:00100000:3.0:1776188903.102409:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c3059999 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664253248:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776188903.102419:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b7c9d478 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664253184:12345-0@lo:13:osp-pre-1-0.0 Request processed in 58us (276us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188903.102427:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6416 00002000:00000024:3.0:1776188903.102432:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776188903.102473:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c3059999 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664253248:12345-0@lo:13:osp-pre-0-0.0 Request processed in 64us (245us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188903.102484:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6417 00000100:00100000:3.0:1776188903.102573:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094ac8433 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664253248:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776188903.102595:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a680c874 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664253184:0@lo:13:osp-pre-1-0.0 00010000:02020000:0.0:1776188903.260331:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188903.516290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188903.772288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188904.028324:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:3.0:1776188904.124740:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188904.124747:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188904.124750:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188904.124758:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9045 00000100:00080000:3.0:1776188904.124761:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776188904.124767:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188904.124789:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e13527ae x1862459664253440/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776188904.124824:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188904.124828:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188904.124830:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188904.124834:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9045 00000100:00080000:3.0:1776188904.124836:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776188904.124839:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188904.124854:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a7368321 x1862459664253696/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776188904.124858:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e13527ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664253440:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776188904.124866:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188904.124869:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188904.124870:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188904.124874:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9045 00000100:00080000:3.0:1776188904.124878:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776188904.124882:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188904.124896:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000053d4c624 x1862459664253760/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776188904.124949:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000050c951fb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664253504:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776188904.124959:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188904.125021:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002046cfde pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664253312:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188904.126213:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a7368321 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664253696:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776188904.126257:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188904.126279:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000053d4c624 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664253760:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776188904.126302:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188904.126343:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664253760 00000100:00100000:1.0:1776188904.126375:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9028 00000100:00100000:1.0:1776188904.126381:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000adb1e111 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664253760:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776188904.127042:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776188904.127099:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000adb1e111 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664253760:12345-0@lo:38:kworker/u8:1.0 Request processed in 717us (795us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188904.127107:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9028 00000100:00080000:1.0:1776188904.127151:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776188904.127155:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776188904.127159:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000053d4c624 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664253760:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776188904.127619:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188904.127646:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000476e266d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664253632:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776188904.127673:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188904.127689:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000085a66d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664254016:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188904.127710:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188904.127748:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664253312 00000100:00100000:0.0:1776188904.127782:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1267 00000100:00100000:0.0:1776188904.127787:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a9501f9f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664253312:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188904.127832:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a9501f9f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664253312:12345-0@lo:400:kworker/u8:1.0 Request processed in 44us (215us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188904.127838:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1267 00000100:00100000:0.0:1776188904.127869:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664253632 00000100:00100000:0.0:1776188904.127895:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6418 00000100:00100000:0.0:1776188904.127899:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000013bfaae8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664253632:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776188904.127915:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776188904.127946:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000013bfaae8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664253632:12345-0@lo:13:osp-pre-0-1.0 Request processed in 45us (272us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188904.127952:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6418 00000100:00100000:0.0:1776188904.128001:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664254016 00000100:00100000:0.0:1776188904.128026:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8237 00000100:00100000:0.0:1776188904.128029:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003fb63a1c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664254016:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188904.128059:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003fb63a1c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664254016:12345-0@lo:400:kworker/u8:1.0 Request processed in 29us (349us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188904.128065:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8237 00000100:00100000:0.0:1776188904.128093:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664253440 00000100:00100000:0.0:1776188904.128114:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664253696 00000100:00100000:0.0:1776188904.128128:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57086 00000100:00100000:0.0:1776188904.128131:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000024430557 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664253440:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776188904.128144:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776188904.128362:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000024430557 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664253440:12345-0@lo:38:kworker/u8:1.0 Request processed in 229us (3405us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188904.128368:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57086 00000100:00100000:0.0:1776188904.128380:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57087 00000100:00100000:0.0:1776188904.128383:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000723457d6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664253696:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776188904.128393:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776188904.128415:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000723457d6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664253696:12345-0@lo:38:kworker/u8:1.0 Request processed in 31us (2160us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188904.128420:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57087 00000100:00100000:0.0:1776188904.128456:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002046cfde pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664253312:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188904.128473:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000476e266d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664253632:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776188904.128481:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000085a66d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664254016:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188904.128551:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b9d589b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664253568:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188904.128580:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188904.128596:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006ad1d61a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664253952:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188904.128617:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188904.128642:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664253568 00000100:00100000:0.0:1776188904.128664:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664253952 00000100:00100000:0.0:1776188904.128677:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9577 00000100:00100000:0.0:1776188904.128680:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba2818a5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664253568:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188904.128713:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba2818a5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664253568:12345-0@lo:400:kworker/u8:1.0 Request processed in 31us (133us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188904.128719:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9577 00000100:00100000:0.0:1776188904.128732:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9578 00000100:00100000:0.0:1776188904.128735:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b29992fd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664253952:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188904.128759:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b29992fd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664253952:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (143us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188904.128764:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9578 00000100:00100000:0.0:1776188904.128788:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b9d589b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664253568:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188904.128797:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006ad1d61a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664253952:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188904.129409:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188904.129482:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002ea4a019 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664253888:0@lo:400:kworker/u8:1.0 00000100:00080000:1.0:1776188904.129499:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776188904.129505:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776188904.129509:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a7368321 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664253696:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776188904.129527:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776188904.129534:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776188904.129538:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776188904.129542:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e13527ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664253440:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776188904.129572:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664253888 00000100:00100000:3.0:1776188904.129602:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8238 00000100:00100000:3.0:1776188904.129607:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b9d589b6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664253888:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188904.129655:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b9d589b6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664253888:12345-0@lo:400:kworker/u8:1.0 Request processed in 48us (140us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188904.129662:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8238 00000100:00100000:1.0:1776188904.129740:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002ea4a019 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664253888:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188904.129785:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664253504 00000100:00100000:0.0:1776188904.129846:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6419 00000100:00100000:0.0:1776188904.129850:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006ad1d61a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664253504:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776188904.129864:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776188904.129907:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006ad1d61a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664253504:12345-0@lo:13:osp-pre-1-1.0 Request processed in 55us (499us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188904.129913:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6419 00000100:00100000:3.0:1776188904.129986:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000050c951fb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664253504:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776188904.132680:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664253376 00000100:00100000:3.0:1776188904.133061:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664253824 00000100:00100000:3.0:1776188904.133093:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664254080 00000100:00100000:0.0:1776188904.133096:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9579 00000100:00100000:0.0:1776188904.133102:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003de93603 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664253376:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188904.133111:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8239 00000100:00100000:3.0:1776188904.133114:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007c9e2a07 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664253824:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188904.133154:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003de93603 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664253376:12345-0@lo:400:kworker/u8:1.0 Request processed in 52us (780us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188904.133154:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007c9e2a07 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664253824:12345-0@lo:400:kworker/u8:1.0 Request processed in 38us (642us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188904.133162:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8239 00000100:00100000:0.0:1776188904.133190:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9579 00000100:00100000:3.0:1776188904.133201:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8240 00000100:00100000:3.0:1776188904.133205:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b50afbbd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664254080:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188904.133245:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b50afbbd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664254080:12345-0@lo:400:kworker/u8:1.0 Request processed in 39us (624us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188904.133254:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8240 00000100:00100000:3.0:1776188904.133307:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000080b4f511 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664253376:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188904.133318:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8a51369 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664253824:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188904.133326:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000b321ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664254080:0@lo:400:kworker/u8:1.0 00010000:02020000:0.0:1776188904.284358:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188904.541144:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188904.796375:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188905.055493:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188905.309192:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188905.564384:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188905.820550:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188906.076645:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188906.332285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188906.589847:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188906.844408:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188907.100420:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188907.356284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188907.612286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188907.868708:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776188907.901086:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776188907.901131:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776188907.901160:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776188907.901265:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776188907.901301:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654932864 00000100:00100000:1.0:1776188907.901360:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654932928 00000100:00100000:1.0:1776188907.901398:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9580 00000100:00100000:1.0:1776188907.901404:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000073262ad2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654932864:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1776188907.901459:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000073262ad2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654932864:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 50us (324us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188907.901479:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9580 00000100:00100000:0.0:1776188907.901500:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57088 00000100:00100000:0.0:1776188907.901508:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cab21754 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654932928:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00000100:00100000:3.0:1776188907.901601:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654932800 00000100:00100000:3.0:1776188907.901637:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1268 00000100:00100000:3.0:1776188907.901642:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004c8f754f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654932800:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1776188907.901682:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004c8f754f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654932800:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 40us (605us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188907.901689:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1268 00010000:02000000:0.0:1776188907.901741:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776188907.901790:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cab21754 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654932928:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 283us (525us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188907.901797:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57088 00010000:02020000:0.0:1776188908.124468:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776188908.220474:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000affce7e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664254208:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776188908.224212:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664254144 00000100:00100000:1.0:1776188908.224281:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6420 00000100:00100000:1.0:1776188908.224287:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022e717c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664254144:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776188908.224318:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776188908.224367:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022e717c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664254144:12345-0@lo:13:osp-pre-1-0.0 Request processed in 80us (701us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188908.224373:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6420 00000100:00100000:3.0:1776188908.225014:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188908.225526:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664254208 00000100:00100000:1.0:1776188908.225613:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6421 00000100:00100000:1.0:1776188908.225619:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000b321ca3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664254208:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776188908.225645:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776188908.225707:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000b321ca3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664254208:12345-0@lo:13:osp-pre-0-0.0 Request processed in 88us (695us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188908.225714:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6421 00000100:00100000:0.0:1776188908.225808:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000affce7e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664254208:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776188908.380297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188908.636744:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188908.892300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188909.148852:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188909.244681:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001f4e478a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664254272:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776188909.244753:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188909.244937:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009522d095 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664254336:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188909.244986:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188909.245024:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad500dff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664254784:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188909.245053:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188909.245077:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1776188909.245085:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f43285e8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664254720:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188909.245114:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188909.245136:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000020eb6bf9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664254976:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188909.245166:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188909.245220:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664254336 00000100:00100000:3.0:1776188909.245267:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1269 00000100:00100000:3.0:1776188909.245274:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a9f52f65 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664254336:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188909.245336:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a9f52f65 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664254336:12345-0@lo:400:kworker/u8:1.0 Request processed in 62us (354us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188909.245344:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1269 00000100:00100000:3.0:1776188909.245379:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664254784 00000100:00100000:3.0:1776188909.245400:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664254720 00000100:00100000:3.0:1776188909.245427:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664254912 00000100:00100000:3.0:1776188909.245441:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664254976 00000100:00100000:3.0:1776188909.245457:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8242 00000100:00100000:3.0:1776188909.245460:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b8a51369 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664254784:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188909.245491:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b8a51369 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664254784:12345-0@lo:400:kworker/u8:1.0 Request processed in 30us (439us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188909.245497:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8242 00000100:00100000:3.0:1776188909.245508:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8243 00000100:00100000:3.0:1776188909.245511:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000080b4f511 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664254720:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188909.245536:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000080b4f511 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664254720:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (424us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188909.245541:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8243 00000100:00100000:3.0:1776188909.245550:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8244 00000100:00100000:3.0:1776188909.245554:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000516ddb7f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664254912:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188909.245576:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000516ddb7f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664254912:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (444us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188909.245582:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8244 00000100:00100000:3.0:1776188909.246155:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664254272 00000100:00100000:0.0:1776188909.246170:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664254528 00000100:00100000:3.0:1776188909.246204:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6422 00000100:00100000:3.0:1776188909.246210:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000affce7e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664254272:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776188909.246224:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664254400 00002000:00000024:3.0:1776188909.246227:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776188909.246239:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664254848 00000100:00100000:0.0:1776188909.246252:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9581 00000100:00100000:0.0:1776188909.246256:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000060073793 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664254528:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188909.246270:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000affce7e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664254272:12345-0@lo:13:osp-pre-0-1.0 Request processed in 60us (1520us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188909.246279:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6422 00000100:00100000:0.0:1776188909.246303:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000060073793 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664254528:12345-0@lo:400:kworker/u8:1.0 Request processed in 44us (1226us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188909.246310:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9581 00000100:00100000:0.0:1776188909.246323:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664254464 00000100:00100000:3.0:1776188909.246329:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f43285e8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664254720:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188909.246333:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664254592 00000100:00100000:3.0:1776188909.246345:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000020eb6bf9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664254976:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188909.246347:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9582 00000100:00100000:0.0:1776188909.246354:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000094a1a2a8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664254400:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188909.246361:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad500dff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664254784:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188909.246373:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009522d095 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664254336:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188909.246391:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000094a1a2a8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664254400:12345-0@lo:400:kworker/u8:1.0 Request processed in 35us (1150us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188909.246398:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9582 00000100:00100000:0.0:1776188909.246422:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9583 00000100:00100000:0.0:1776188909.246427:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007d51703e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664254848:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188909.246435:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001f4e478a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664254272:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776188909.246457:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007d51703e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664254848:12345-0@lo:400:kworker/u8:1.0 Request processed in 30us (1179us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188909.246465:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9583 00000100:00100000:3.0:1776188909.246482:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e42367aa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664254400:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188909.246492:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000408e2292 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664254848:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188909.246501:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed846a61 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664254528:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188909.246882:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57089 00000100:00100000:3.0:1776188909.246887:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001feee8d8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664254464:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776188909.246895:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188909.246935:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001feee8d8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664254464:12345-0@lo:38:kworker/u8:1.0 Request processed in 46us (1252us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188909.246947:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57089 00000100:00100000:3.0:1776188909.247063:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57090 00000100:00100000:3.0:1776188909.247070:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001680f253 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664254592:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776188909.247082:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188909.247119:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001680f253 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664254592:12345-0@lo:38:kworker/u8:1.0 Request processed in 48us (1407us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188909.247129:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57090 00010000:02020000:0.0:1776188909.404336:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188909.660304:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188909.916402:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188910.172844:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188910.428287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188910.684434:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188910.940290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188911.196877:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188911.452313:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188911.708232:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188911.964264:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188912.220290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188912.476802:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188912.732288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188912.988220:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188913.022264:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654933056 00000100:00100000:3.0:1776188913.022313:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654933120 00000100:00100000:0.0:1776188913.022347:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1270 00000100:00100000:0.0:1776188913.022353:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000be36d03e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654933056:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1776188913.022363:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654933184 00000100:00100000:3.0:1776188913.022388:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9584 00000100:00100000:0.0:1776188913.022413:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000be36d03e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654933056:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 58us (1950us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188913.022421:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1270 00000100:00100000:0.0:1776188913.022455:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57091 00000100:00100000:0.0:1776188913.022459:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000981bac76 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654933184:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:0.0:1776188913.022474:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776188913.022500:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000981bac76 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654933184:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 40us (2181us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188913.022506:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57091 00000100:00100000:3.0:1776188913.023754:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004fbf075f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654933120:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1776188913.024081:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004fbf075f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654933120:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 1686us (3814us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188913.024091:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9584 00010000:02020000:0.0:1776188913.245098:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776188913.341224:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776188913.341245:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002046cfde pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664255104:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776188913.341307:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188913.341361:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664255104 00000100:00100000:3.0:1776188913.341400:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6424 00000100:00100000:3.0:1776188913.341405:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053d4c624 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664255104:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776188913.341432:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776188913.341471:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053d4c624 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664255104:12345-0@lo:13:osp-pre-1-0.0 Request processed in 64us (164us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188913.341478:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6424 00000100:00100000:3.0:1776188913.341523:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002046cfde pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664255104:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776188913.347125:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664255168 00000100:00100000:1.0:1776188913.347215:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6425 00000100:00100000:1.0:1776188913.347221:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ebbbbd87 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664255168:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776188913.347254:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776188913.347302:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ebbbbd87 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664255168:12345-0@lo:13:osp-pre-0-0.0 Request processed in 79us (654us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188913.347309:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6425 00010000:02020000:0.0:1776188913.500224:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188913.756541:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188914.012290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188914.268447:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:0.0:1776188914.365020:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776188914.365025:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776188914.365028:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776188914.365033:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9056 00000100:00080000:0.0:1776188914.365036:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00080000:0.0:1776188914.365041:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776188914.365058:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e139d197 x1862459664255488/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776188914.365080:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776188914.365083:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776188914.365084:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776188914.365087:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9056 00000100:00080000:0.0:1776188914.365089:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:0.0:1776188914.365093:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776188914.365102:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002c842ea9 x1862459664255616/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776188914.365109:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776188914.365112:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776188914.365113:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776188914.365116:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9056 00000100:00080000:0.0:1776188914.365117:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 11s 00000100:00080000:0.0:1776188914.365119:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776188914.365128:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000046fee2b x1862459664255680/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776188914.365207:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664255296 00000100:00100000:0.0:1776188914.365208:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000914d0a5f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664255424:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188914.365248:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188914.365249:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664255232 00000100:00100000:0.0:1776188914.365271:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000040b53b34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664255872:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188914.365276:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6426 00000100:00100000:3.0:1776188914.365306:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b50afbbd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664255296:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776188914.365307:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188914.365325:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1776188914.365331:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fdda22fa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664255360:0@lo:400:kworker/u8:0.0 00002000:00000024:3.0:1776188914.365332:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776188914.365435:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001b6b6588 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664255552:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188914.365528:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b50afbbd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664255296:12345-0@lo:13:osp-pre-0-1.0 Request processed in 233us (873us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188914.365536:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6426 00000100:00100000:3.0:1776188914.365554:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6427 00000100:00100000:3.0:1776188914.365557:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000516ddb7f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664255232:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776188914.365642:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776188914.365672:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000516ddb7f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664255232:12345-0@lo:13:osp-pre-1-1.0 Request processed in 114us (933us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188914.365678:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6427 00000100:00100000:3.0:1776188914.365702:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664255424 00000100:00100000:3.0:1776188914.365729:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664255872 00000100:00100000:3.0:1776188914.365745:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9585 00000100:00100000:3.0:1776188914.365749:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000673b41e8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664255424:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188914.365779:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000673b41e8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664255424:12345-0@lo:400:kworker/u8:0.0 Request processed in 29us (532us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188914.365785:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9585 00000100:00100000:3.0:1776188914.365795:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9586 00000100:00100000:3.0:1776188914.365798:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000061dd8d04 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664255872:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188914.365822:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000061dd8d04 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664255872:12345-0@lo:400:kworker/u8:0.0 Request processed in 22us (527us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188914.365826:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9586 00000100:00100000:3.0:1776188914.365860:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c1af713e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664255744:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188914.365887:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188914.365937:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002046cfde pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664255232:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776188914.365964:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007c9e2a07 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664255296:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776188914.365997:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664255744 00000100:00100000:3.0:1776188914.366021:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8247 00000100:00100000:3.0:1776188914.366035:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000528f790 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664255744:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188914.366066:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000528f790 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664255744:12345-0@lo:400:kworker/u8:0.0 Request processed in 30us (179us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188914.366072:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8247 00000100:00100000:3.0:1776188914.366097:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c1af713e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664255744:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188914.371137:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188914.371508:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664255488 00000100:00100000:1.0:1776188914.371558:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57092 00000100:00100000:1.0:1776188914.371564:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007c9e2a07 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664255488:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776188914.371584:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188914.373489:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664255360 00000100:00100000:3.0:1776188914.374047:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1271 00000100:00100000:3.0:1776188914.374053:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000816fca97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664255360:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188914.376122:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000205cde5a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664255808:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188914.376239:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188914.376280:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664255808 00000100:00100000:0.0:1776188914.376336:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8248 00000100:00100000:0.0:1776188914.376340:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000aafe5128 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664255808:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188914.376381:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000aafe5128 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664255808:12345-0@lo:400:kworker/u8:0.0 Request processed in 40us (144us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188914.376388:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8248 00000100:00100000:0.0:1776188914.376425:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000914d0a5f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664255424:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188914.376435:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000040b53b34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664255872:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188914.376458:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000205cde5a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664255808:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188914.376492:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188914.376531:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664255552 00000100:00100000:0.0:1776188914.376555:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9587 00000100:00100000:0.0:1776188914.376559:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000205cde5a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664255552:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188914.376596:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000205cde5a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664255552:12345-0@lo:400:kworker/u8:0.0 Request processed in 37us (104us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188914.376602:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9587 00000100:00100000:0.0:1776188914.376628:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001b6b6588 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664255552:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188914.377607:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007c9e2a07 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664255488:12345-0@lo:38:kworker/u8:0.0 Request processed in 6038us (6218us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188914.377618:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57092 00000100:00100000:2.0:1776188914.377730:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188914.377746:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000046fee2b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664255680:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776188914.377770:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1776188914.377862:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776188914.377867:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776188914.377871:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e139d197 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664255488:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776188914.377920:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664255680 00000100:00100000:1.0:1776188914.377946:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664255616 00000100:00100000:1.0:1776188914.377975:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57093 00000100:00100000:1.0:1776188914.377980:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ed846a61 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664255616:12345-0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776188914.377987:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9030 00000100:00100000:0.0:1776188914.377992:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000408e2292 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664255680:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776188914.377997:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00010000:02000000:0.0:1776188914.378005:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00080000:2.0:1776188914.378114:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776188914.378117:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776188914.378126:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002c842ea9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664255616:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776188914.378396:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ed846a61 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664255616:12345-0@lo:38:kworker/u8:0.0 Request processed in 414us (665us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188914.378403:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57093 00000100:00100000:0.0:1776188914.378536:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000408e2292 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664255680:12345-0@lo:38:kworker/u8:0.0 Request processed in 541us (764us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188914.378543:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9030 00000100:00100000:3.0:1776188914.378575:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000816fca97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664255360:12345-0@lo:400:kworker/u8:0.0 Request processed in 4519us (7442us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188914.378587:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1271 00000100:00100000:1.0:1776188914.378611:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fdda22fa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664255360:0@lo:400:kworker/u8:0.0 00000100:00080000:2.0:1776188914.378813:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776188914.378817:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776188914.378820:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000046fee2b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664255680:0@lo:38:kworker/u8:0.0 00010000:02020000:0.0:1776188914.524510:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188914.686696:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188914.780931:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188915.036296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188915.292329:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188915.548500:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188915.708409:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188915.804669:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188916.061643:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188916.317846:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188916.572909:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188916.733339:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188916.828325:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188917.084323:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188917.340334:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188917.596809:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188917.756426:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188917.852349:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188918.109134:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776188918.143087:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188918.143336:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654933312 00000100:00100000:3.0:1776188918.143395:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1272 00000100:00100000:3.0:1776188918.143402:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c4f71e57 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654933312:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776188918.143481:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c4f71e57 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654933312:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 78us (404us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188918.143489:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1272 00000100:00100000:2.0:1776188918.143628:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776188918.143888:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654933376 00000100:00100000:1.0:1776188918.143928:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9588 00000100:00100000:1.0:1776188918.143932:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e42367aa pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654933376:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776188918.143982:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e42367aa pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654933376:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 48us (355us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188918.143988:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9588 00000100:00100000:2.0:1776188918.145563:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188918.145601:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188918.145697:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654933504 00000100:00100000:0.0:1776188918.145743:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8249 00000100:00100000:0.0:1776188918.145748:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009522d095 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2280:x1862459654933504:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776188918.145811:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009522d095 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2280:x1862459654933504:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 62us (253us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188918.145818:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8249 00000100:00100000:1.0:1776188918.146272:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654933440 00000100:00100000:1.0:1776188918.146307:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57094 00000100:00100000:1.0:1776188918.146311:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ad500dff pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654933440:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776188918.146326:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776188918.146370:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ad500dff pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654933440:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 58us (768us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188918.146376:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57094 00010000:02020000:0.0:1776188918.364829:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776188918.461110:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776188918.461132:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a680c874 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664255936:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776188918.461224:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188918.461318:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664255936 00000100:00100000:2.0:1776188918.461368:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6428 00000100:00100000:2.0:1776188918.461373:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002046cfde pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664255936:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776188918.461401:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776188918.461445:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002046cfde pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664255936:12345-0@lo:13:osp-pre-0-0.0 Request processed in 71us (222us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188918.461456:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6428 00000100:00100000:0.0:1776188918.461580:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776188918.461591:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d2d0e609 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664256000:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776188918.461621:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188918.464984:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664256000 00000100:00100000:2.0:1776188918.465080:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6429 00000100:00100000:2.0:1776188918.465087:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001b6b6588 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664256000:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776188918.465122:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776188918.466490:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a680c874 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664255936:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776188918.466518:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001b6b6588 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664256000:12345-0@lo:13:osp-pre-1-0.0 Request processed in 1430us (4896us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188918.466527:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6429 00000100:00100000:1.0:1776188918.467078:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d2d0e609 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664256000:0@lo:13:osp-pre-1-0.0 00010000:02020000:0.0:1776188918.620292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188918.785164:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188918.876342:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188919.133915:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188919.388301:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776188919.486393:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a388dd2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664256064:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776188919.486478:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188919.486493:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000020eb6bf9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664256128:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776188919.486531:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188919.486580:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664256064 00000100:00100000:0.0:1776188919.486625:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664256128 00000100:00100000:0.0:1776188919.486644:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6430 00000100:00100000:0.0:1776188919.486651:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d2d0e609 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664256064:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776188919.486665:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6431 00002000:00000024:0.0:1776188919.486674:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776188919.486715:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d2d0e609 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664256064:12345-0@lo:13:osp-pre-1-1.0 Request processed in 64us (238us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188919.486722:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6430 00000100:00100000:1.0:1776188919.486804:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a388dd2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664256064:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776188919.486816:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f43285e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664256128:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776188919.486834:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00080000:3.0:1776188919.486920:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188919.486925:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188919.486929:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188919.486933:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9061 00000100:00080000:3.0:1776188919.486936:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1776188919.486939:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188919.486956:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b8a51369 x1862459664256320/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776188919.486981:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188919.486985:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188919.486986:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1776188919.486990:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f43285e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664256128:12345-0@lo:13:osp-pre-0-1.0 Request processed in 316us (458us total) trans 0 rc 0/0 00000100:00080000:3.0:1776188919.486993:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9061 00000100:00080000:3.0:1776188919.486997:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00100000:2.0:1776188919.487003:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6431 00000100:00080000:3.0:1776188919.487004:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188919.487018:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000085a66d x1862459664256448/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776188919.487027:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1776188919.487031:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8a51369 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664256320:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776188919.487032:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188919.487034:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188919.487040:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9061 00000100:00080000:3.0:1776188919.487042:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1776188919.487044:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188919.487057:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002ea4a019 x1862459664256512/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776188919.487068:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188919.487094:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000085a66d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664256448:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188919.487114:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e13527ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664256576:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188919.487121:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188919.487138:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002ea4a019 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664256512:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188919.487154:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188919.487203:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188919.487219:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000020eb6bf9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664256128:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776188919.487242:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776188919.487243:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664256512 00000100:00100000:3.0:1776188919.487248:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000080b4f511 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664256384:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188919.487268:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9031 00000100:00100000:2.0:1776188919.487273:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d97d7d85 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664256512:12345-0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188919.487279:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:02000000:2.0:1776188919.487290:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188919.487308:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664256576 00000100:00100000:2.0:1776188919.487330:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d97d7d85 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664256512:12345-0@lo:38:kworker/u8:0.0 Request processed in 54us (159us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776188919.487338:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9031 00000100:00100000:3.0:1776188919.487343:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8250 00000100:00100000:3.0:1776188919.487349:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005ed2a569 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664256576:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188919.487365:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664256320 00000100:00100000:2.0:1776188919.487388:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664256448 00000100:00100000:3.0:1776188919.487391:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005ed2a569 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664256576:12345-0@lo:400:kworker/u8:0.0 Request processed in 41us (238us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188919.487398:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8250 00000100:00100000:2.0:1776188919.487405:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664256384 00000100:00100000:2.0:1776188919.487432:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9589 00000100:00100000:2.0:1776188919.487436:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c30e127a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664256384:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188919.487468:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b9d589b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664256256:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188919.487469:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c30e127a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664256384:12345-0@lo:400:kworker/u8:0.0 Request processed in 31us (192us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188919.487480:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9589 00000100:00100000:2.0:1776188919.487495:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57095 00000100:00100000:2.0:1776188919.487499:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000079a89d96 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664256320:12345-0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776188919.487504:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:02000000:2.0:1776188919.487508:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776188919.487534:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000079a89d96 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664256320:12345-0@lo:38:kworker/u8:0.0 Request processed in 34us (467us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776188919.487541:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57095 00000100:00100000:1.0:1776188919.487560:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007390d733 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664256192:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188919.487597:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188919.487614:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a7368321 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664256640:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188919.487634:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188919.491066:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664256256 00000100:00100000:2.0:1776188919.491114:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9590 00000100:00100000:2.0:1776188919.491120:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000040b53b34 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664256256:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188919.491128:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a388dd2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664256704:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188919.491215:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57096 00000100:00100000:3.0:1776188919.491221:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000058e0433d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664256448:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776188919.491237:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776188919.491274:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000040b53b34 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664256256:12345-0@lo:400:kworker/u8:0.0 Request processed in 152us (3770us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188919.491281:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000058e0433d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664256448:12345-0@lo:38:kworker/u8:0.0 Request processed in 58us (4160us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776188919.491284:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9590 00000100:00100000:3.0:1776188919.491294:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57096 00000100:00100000:3.0:1776188919.491329:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664256640 00000100:00080000:2.0:1776188919.491339:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776188919.491345:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776188919.491349:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000085a66d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664256448:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776188919.491363:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:3.0:1776188919.491363:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8251 00000100:00080000:2.0:1776188919.491368:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188919.491371:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a680c874 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664256640:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188919.491374:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002ea4a019 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664256512:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776188919.491386:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776188919.491389:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776188919.491392:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8a51369 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664256320:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188919.491422:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a680c874 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664256640:12345-0@lo:400:kworker/u8:0.0 Request processed in 52us (3787us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188919.491430:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8251 00000100:00100000:2.0:1776188919.491448:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664256192 00000100:00100000:3.0:1776188919.491469:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a7368321 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664256640:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188919.491479:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1273 00000100:00100000:2.0:1776188919.491485:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b1a6ecdc pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664256192:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188919.491520:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000080b4f511 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664256384:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188919.491526:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b1a6ecdc pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664256192:12345-0@lo:400:kworker/u8:0.0 Request processed in 40us (3928us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188919.491534:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1273 00000100:00100000:3.0:1776188919.491537:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e13527ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664256576:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188919.491543:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188919.491574:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007390d733 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664256192:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188919.491586:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b9d589b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664256256:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188919.491592:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664256704 00000100:00100000:2.0:1776188919.491623:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9591 00000100:00100000:2.0:1776188919.491627:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000914d0a5f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664256704:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188919.491661:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000914d0a5f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664256704:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (121us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188919.491667:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9591 00000100:00100000:0.0:1776188919.496429:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a388dd2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664256704:0@lo:400:kworker/u8:0.0 00010000:02020000:0.0:1776188919.644722:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188919.805834:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188919.900828:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188920.156694:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188920.412296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188920.668299:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188920.828623:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188920.928328:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188921.180295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188921.436702:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188921.693466:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188921.853519:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188921.949476:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188922.205377:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188922.460281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188922.716449:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188922.878088:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188922.972353:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188923.229206:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776188923.263160:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188923.263251:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188923.263285:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188923.263322:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188923.263355:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654933632 00000100:00100000:1.0:1776188923.263377:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654933696 00000100:00100000:2.0:1776188923.263408:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9592 00000100:00100000:2.0:1776188923.263416:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007a388dd2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654933632:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1776188923.263422:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57097 00000100:00100000:1.0:1776188923.263429:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007390d733 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654933696:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:1.0:1776188923.263445:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776188923.263526:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654933760 00000100:00100000:3.0:1776188923.263567:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8252 00000100:00100000:3.0:1776188923.263571:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b9d589b6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654933760:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1776188923.263614:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007a388dd2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654933632:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 70us (334us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188923.263622:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9592 00000100:00100000:0.0:1776188923.263682:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654933568 00000100:00100000:0.0:1776188923.263708:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1274 00000100:00100000:0.0:1776188923.263712:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000090683405 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654933568:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1776188923.265109:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007390d733 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654933696:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 1664us (1772us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188923.265123:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57097 00000100:00100000:3.0:1776188923.266158:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b9d589b6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654933760:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 2581us (2868us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188923.266192:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8252 00000100:00100000:0.0:1776188923.266917:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000090683405 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654933568:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 3201us (3664us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188923.266928:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1274 00010000:02020000:1.0:1776188923.484329:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776188923.580415:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c68f859e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664256768:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776188923.580477:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188923.580564:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664256768 00000100:00100000:1.0:1776188923.580613:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6432 00000100:00100000:1.0:1776188923.580619:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e13527ae pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664256768:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776188923.580646:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776188923.580685:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e13527ae pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664256768:12345-0@lo:13:osp-pre-1-0.0 Request processed in 66us (209us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188923.580691:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6432 00000100:00100000:2.0:1776188923.580772:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c68f859e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664256768:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776188923.581125:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024c95b46 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664256832:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776188923.581154:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188923.581211:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664256832 00000100:00100000:2.0:1776188923.581237:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6433 00000100:00100000:2.0:1776188923.581241:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000020eb6bf9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664256832:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776188923.581258:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776188923.582865:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000020eb6bf9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664256832:12345-0@lo:13:osp-pre-0-0.0 Request processed in 1618us (1707us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188923.582878:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6433 00000100:00100000:3.0:1776188923.583030:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024c95b46 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664256832:0@lo:13:osp-pre-0-0.0 00010000:02020000:1.0:1776188923.740997:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188923.900803:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188923.997499:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188924.252328:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188924.508333:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:3.0:1776188924.604966:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188924.604973:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188924.604976:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188924.604984:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9066 00000100:00080000:3.0:1776188924.604989:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776188924.604994:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188924.605013:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000549bc579 x1862459664257024/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776188924.605045:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188924.605049:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188924.605050:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188924.605053:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9066 00000100:00080000:3.0:1776188924.605055:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776188924.605058:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188924.605068:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b5f1e324 x1862459664257152/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776188924.605086:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188924.605089:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188924.605090:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188924.605093:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9066 00000100:00080000:3.0:1776188924.605095:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776188924.605098:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188924.605112:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e6665a35 x1862459664257216/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776188924.605164:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000031aca57e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664257280:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188924.605239:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188924.605265:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fdab0c5c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664257536:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188924.605286:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188924.605305:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1776188924.605309:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000576bf6bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664257088:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188924.605330:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188924.605342:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ccb5377d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664257472:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188924.605359:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188924.605392:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664257280 00000100:00100000:3.0:1776188924.605429:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664257536 00000100:00100000:3.0:1776188924.605446:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664257472 00000100:00100000:3.0:1776188924.605463:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8253 00000100:00100000:3.0:1776188924.605469:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d9b9c94e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664257280:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188924.605512:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d9b9c94e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664257280:12345-0@lo:400:kworker/u8:0.0 Request processed in 42us (273us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188924.605519:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8253 00000100:00100000:3.0:1776188924.605537:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8254 00000100:00100000:3.0:1776188924.605540:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b15f52c9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664257536:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188924.605564:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b15f52c9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664257536:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (278us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188924.605569:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8254 00000100:00100000:3.0:1776188924.605578:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8255 00000100:00100000:3.0:1776188924.605581:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b64538ad pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664257472:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188924.605602:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b64538ad pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664257472:12345-0@lo:400:kworker/u8:0.0 Request processed in 20us (243us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188924.605607:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8255 00000100:00100000:3.0:1776188924.605653:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ccb5377d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664257472:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188924.605663:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000031aca57e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664257280:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188924.605670:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fdab0c5c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664257536:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188924.606528:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4a5ce06 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664256896:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188924.606589:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188924.606620:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003b5d789a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664257344:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188924.606641:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188924.606661:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1776188924.606666:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bd5316a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664256960:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188924.606688:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188924.606701:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005bd91e44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664257408:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188924.606718:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188924.606747:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664257344 00000100:00100000:0.0:1776188924.606774:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8256 00000100:00100000:0.0:1776188924.606778:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009e6a2c02 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664257344:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188924.606819:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009e6a2c02 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664257344:12345-0@lo:400:kworker/u8:0.0 Request processed in 40us (177us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188924.606825:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8256 00000100:00100000:0.0:1776188924.606855:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003b5d789a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664257344:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188924.607235:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664257088 00000100:00100000:1.0:1776188924.607262:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664256960 00000100:00100000:1.0:1776188924.607279:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664257408 00000100:00100000:1.0:1776188924.607293:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9593 00000100:00100000:1.0:1776188924.607297:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d82e179d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664257088:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188924.607348:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d82e179d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664257088:12345-0@lo:400:kworker/u8:0.0 Request processed in 50us (2018us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188924.607355:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9593 00000100:00100000:1.0:1776188924.607367:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9594 00000100:00100000:1.0:1776188924.607371:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000acf34144 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664256960:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188924.607400:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000acf34144 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664256960:12345-0@lo:400:kworker/u8:0.0 Request processed in 28us (712us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188924.607406:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9594 00000100:00100000:1.0:1776188924.607417:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9595 00000100:00100000:1.0:1776188924.607419:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003fb63a1c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664257408:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188924.607443:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003fb63a1c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664257408:12345-0@lo:400:kworker/u8:0.0 Request processed in 22us (724us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188924.607448:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9595 00000100:00100000:1.0:1776188924.607481:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bd5316a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664256960:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188924.607490:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005bd91e44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664257408:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188924.607511:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000576bf6bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664257088:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188924.608048:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000549bc579 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664257024:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776188924.608091:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188924.608108:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5f1e324 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664257152:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776188924.608128:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188924.608140:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6665a35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664257216:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776188924.608162:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188924.608217:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664257024 00000100:00100000:2.0:1776188924.608239:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664257152 00000100:00100000:2.0:1776188924.608249:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57098 00000100:00100000:2.0:1776188924.608253:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a7368321 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664257024:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776188924.608267:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776188924.608300:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a7368321 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664257024:12345-0@lo:38:kworker/u8:0.0 Request processed in 46us (210us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776188924.608307:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57098 00000100:00100000:2.0:1776188924.608332:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57099 00000100:00100000:2.0:1776188924.608335:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b8a51369 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664257152:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776188924.608343:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776188924.608364:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b8a51369 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664257152:12345-0@lo:38:kworker/u8:0.0 Request processed in 28us (236us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776188924.608369:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57099 00000100:00100000:2.0:1776188924.608382:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776188924.608390:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000080b4f511 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664257600:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776188924.608415:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188924.608442:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664257216 00000100:00100000:2.0:1776188924.608460:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9032 00000100:00100000:2.0:1776188924.608463:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002ea4a019 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664257216:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776188924.608470:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776188924.608492:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002ea4a019 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664257216:12345-0@lo:38:kworker/u8:0.0 Request processed in 28us (330us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776188924.608497:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9032 00000100:00100000:2.0:1776188924.608522:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664257600 00000100:00100000:2.0:1776188924.608558:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6434 00000100:00100000:2.0:1776188924.608563:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000085a66d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664257600:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776188924.608578:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776188924.608610:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000085a66d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664257600:12345-0@lo:13:osp-pre-0-1.0 Request processed in 47us (196us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188924.608616:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6434 00000100:00100000:2.0:1776188924.608649:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000080b4f511 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664257600:0@lo:13:osp-pre-0-1.0 00000100:00080000:2.0:1776188924.608674:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776188924.608678:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776188924.608682:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000549bc579 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664257024:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776188924.608689:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776188924.608691:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776188924.608694:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5f1e324 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664257152:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776188924.608699:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776188924.608702:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776188924.608704:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6665a35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664257216:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188924.609759:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664256896 00000100:00100000:3.0:1776188924.609807:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1275 00000100:00100000:3.0:1776188924.609811:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b90a26ba pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664256896:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188924.609861:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b90a26ba pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664256896:12345-0@lo:400:kworker/u8:0.0 Request processed in 49us (3272us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188924.609867:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1275 00000100:00100000:1.0:1776188924.609978:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4a5ce06 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664256896:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188924.609995:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776188924.610002:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094ac8433 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664257664:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776188924.610030:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188924.610202:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664257664 00000100:00100000:3.0:1776188924.610234:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6435 00000100:00100000:3.0:1776188924.610238:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000020ebaeac pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664257664:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776188924.610253:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776188924.610284:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000020ebaeac pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664257664:12345-0@lo:13:osp-pre-1-1.0 Request processed in 45us (254us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188924.610290:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6435 00000100:00100000:1.0:1776188924.610681:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094ac8433 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664257664:0@lo:13:osp-pre-1-1.0 00010000:02020000:1.0:1776188924.765453:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188924.924534:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188925.020293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188925.276299:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188925.532307:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188925.788278:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188925.949380:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188926.044281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188926.300288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188926.556284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188926.812297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188926.972759:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188927.068442:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188927.324300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188927.580499:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188927.836562:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188927.996592:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188928.093061:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188928.348292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776188928.379635:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188928.381460:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188928.381501:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188928.381580:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188928.381603:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654934016 00000100:00100000:2.0:1776188928.381614:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654933952 00000100:00100000:2.0:1776188928.381652:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654933888 00000100:00100000:3.0:1776188928.381657:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8257 00000100:00100000:3.0:1776188928.381664:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000087b82cd7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654934016:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776188928.381693:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9596 00000100:00100000:2.0:1776188928.381699:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f0b13a19 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654933888:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776188928.381734:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000087b82cd7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654934016:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 69us (232us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188928.381739:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f0b13a19 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654933888:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 36us (157us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188928.381746:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8257 00000100:00100000:2.0:1776188928.381750:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9596 00000100:00100000:2.0:1776188928.381770:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57100 00000100:00100000:2.0:1776188928.381774:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000080b4f511 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654933952:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776188928.381789:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776188928.381818:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000080b4f511 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654933952:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 43us (2191us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776188928.381824:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57100 00000100:00100000:0.0:1776188928.381919:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654933824 00000100:00100000:0.0:1776188928.381949:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1276 00000100:00100000:0.0:1776188928.381953:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ef3ea18a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654933824:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776188928.382008:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ef3ea18a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654933824:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 53us (555us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188928.382014:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1276 00010000:02020000:1.0:1776188928.604372:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776188928.702453:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028933edf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664257728:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776188928.702533:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188928.702769:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664257728 00000100:00100000:1.0:1776188928.702819:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6436 00000100:00100000:1.0:1776188928.702824:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005d995c27 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664257728:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776188928.702855:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776188928.702908:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005d995c27 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664257728:12345-0@lo:13:osp-pre-1-0.0 Request processed in 83us (376us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188928.702914:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6436 00000100:00100000:2.0:1776188928.703000:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028933edf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664257728:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776188928.703832:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776188928.703846:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094ac8433 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664257792:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776188928.703899:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188928.704022:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664257792 00000100:00100000:3.0:1776188928.704063:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6437 00000100:00100000:3.0:1776188928.704068:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000960dca04 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664257792:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776188928.704091:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776188928.704138:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000960dca04 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664257792:12345-0@lo:13:osp-pre-0-0.0 Request processed in 69us (240us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188928.704145:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6437 00000100:00100000:2.0:1776188928.704254:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094ac8433 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664257792:0@lo:13:osp-pre-0-0.0 00010000:02020000:1.0:1776188928.860360:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188929.021521:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188929.116917:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188929.372450:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188929.628284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776188929.724371:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776188929.724410:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3b3d75f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664257856:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776188929.724485:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188929.724567:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664257856 00000100:00100000:2.0:1776188929.724617:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6438 00000100:00100000:2.0:1776188929.724623:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000037fc1275 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664257856:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776188929.724645:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00080000:3.0:1776188929.724682:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188929.724688:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1776188929.724692:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000037fc1275 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664257856:12345-0@lo:13:osp-pre-0-1.0 Request processed in 67us (207us total) trans 0 rc 0/0 00000100:00080000:3.0:1776188929.724692:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188929.724701:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9071 00000100:00100000:2.0:1776188929.724704:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6438 00000100:00080000:3.0:1776188929.724705:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1776188929.724743:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d2b21909 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664257984:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188929.724768:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188929.724794:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664257984 00000100:00100000:2.0:1776188929.724820:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9597 00000100:00100000:2.0:1776188929.724824:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c731e56f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664257984:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188929.724856:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c731e56f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664257984:12345-0@lo:400:kworker/u8:0.0 Request processed in 31us (87us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188929.724871:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9597 00000100:00100000:2.0:1776188929.724905:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d2b21909 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664257984:0@lo:400:kworker/u8:0.0 00000100:00080000:3.0:1776188929.724932:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776188929.724941:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776188929.724960:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000006dfa37f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664257920:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188929.724976:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c4620d7d x1862459664258112/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776188929.724987:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776188929.725000:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024c95b46 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664258048:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776188929.725002:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1776188929.725013:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188929.725017:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188929.725019:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188929.725024:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9071 00000100:00080000:3.0:1776188929.725026:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1776188929.725031:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a0ce2a1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664258176:0@lo:400:kworker/u8:0.0 00000100:00080000:3.0:1776188929.725032:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776188929.725039:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188929.725048:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c0bb1b99 x1862459664258240/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776188929.725068:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1776188929.725068:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1776188929.725073:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664258048 00000100:00080000:3.0:1776188929.725073:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188929.725076:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188929.725081:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9071 00000100:00080000:3.0:1776188929.725083:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776188929.725086:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188929.725101:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d2b21909 x1862459664258304/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776188929.725109:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6439 00000100:00100000:0.0:1776188929.725115:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000046fee2b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664258048:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776188929.725120:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3b3d75f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664257856:0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776188929.725129:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776188929.725140:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6665a35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664258368:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188929.725164:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003b5d789a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664258496:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188929.725169:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000046fee2b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664258048:12345-0@lo:13:osp-pre-1-1.0 Request processed in 53us (132us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188929.725201:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188929.725204:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6439 00000100:00100000:3.0:1776188929.725224:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188929.725249:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664257920 00000100:00100000:3.0:1776188929.725254:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776188929.725261:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005bd91e44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664258432:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188929.725277:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1277 00000100:00100000:0.0:1776188929.725282:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005c923f64 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664257920:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188929.725289:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188929.725323:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005c923f64 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664257920:12345-0@lo:400:kworker/u8:0.0 Request processed in 40us (325us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188929.725331:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1277 00000100:00100000:3.0:1776188929.725335:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664258368 00000100:00100000:0.0:1776188929.725360:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c4620d7d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664258112:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188929.725368:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664258432 00000100:00100000:3.0:1776188929.725388:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8258 00000100:00100000:3.0:1776188929.725392:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b1d7bdea pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664258368:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188929.725415:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664258176 00000100:00100000:2.0:1776188929.725447:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664258496 00000100:00100000:2.0:1776188929.725465:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9598 00000100:00100000:2.0:1776188929.725469:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000229fc20a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664258176:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188929.725615:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8259 00000100:00100000:1.0:1776188929.725620:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fdab0c5c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664258432:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188929.728366:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b1d7bdea pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664258368:12345-0@lo:400:kworker/u8:0.0 Request processed in 2965us (3159us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188929.728383:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8258 00000100:00100000:3.0:1776188929.728443:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9599 00000100:00100000:3.0:1776188929.728450:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000031aca57e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664258496:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188929.728507:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000031aca57e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664258496:12345-0@lo:400:kworker/u8:0.0 Request processed in 58us (3284us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188929.728513:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9599 00000100:00100000:3.0:1776188929.728556:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003b5d789a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664258496:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188929.728598:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024c95b46 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664258048:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776188929.728628:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6665a35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664258368:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188929.728699:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000006dfa37f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664257920:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188929.729036:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188929.729094:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fdab0c5c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664258432:12345-0@lo:400:kworker/u8:0.0 Request processed in 3472us (3807us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188929.729115:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8259 00000100:00100000:1.0:1776188929.729150:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005bd91e44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664258432:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188929.729660:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000229fc20a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664258176:12345-0@lo:400:kworker/u8:0.0 Request processed in 4188us (4593us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188929.729670:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9598 00000100:00100000:0.0:1776188929.729723:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c0bb1b99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664258240:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776188929.729755:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188929.729787:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664258112 00000100:00100000:2.0:1776188929.729853:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57101 00000100:00100000:2.0:1776188929.729858:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fdda22fa pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664258112:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776188929.729879:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776188929.729907:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fdda22fa pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664258112:12345-0@lo:38:kworker/u8:0.0 Request processed in 48us (873us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776188929.729912:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57101 00000100:00100000:0.0:1776188929.730065:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d2b21909 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664258304:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776188929.730093:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776188929.730128:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776188929.730131:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776188929.730135:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c4620d7d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664258112:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776188929.730154:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664258304 00000100:00100000:0.0:1776188929.730186:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9033 00000100:00100000:0.0:1776188929.730190:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003d966fd0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664258304:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776188929.730197:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776188929.730221:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003d966fd0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664258304:12345-0@lo:38:kworker/u8:0.0 Request processed in 31us (129us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188929.730239:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9033 00000100:00100000:1.0:1776188929.730644:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664258240 00000100:00100000:1.0:1776188929.730671:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57102 00000100:00100000:1.0:1776188929.730674:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002c842ea9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664258240:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776188929.730682:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776188929.730708:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002c842ea9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664258240:12345-0@lo:38:kworker/u8:0.0 Request processed in 33us (953us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188929.730714:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57102 00000100:00080000:1.0:1776188929.730754:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776188929.730758:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776188929.730761:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c0bb1b99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664258240:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776188929.730770:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776188929.730772:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776188929.730774:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d2b21909 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664258304:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188929.737410:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a0ce2a1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664258176:0@lo:400:kworker/u8:0.0 00010000:02020000:1.0:1776188929.884854:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188930.052895:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188930.140281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188930.396893:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188930.652296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188930.908468:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188931.069572:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188931.164309:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188931.420289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188931.677389:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188931.932763:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188932.092424:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188932.188271:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188932.447733:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188932.700366:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188932.957361:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188933.116718:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188933.212296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188933.468273:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776188933.499918:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188933.499968:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188933.500051:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188933.500095:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188933.500126:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654934272 00000100:00100000:2.0:1776188933.500210:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8260 00000100:00100000:2.0:1776188933.500217:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ea0541f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654934272:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776188933.500291:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ea0541f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654934272:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 73us (196us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188933.500299:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8260 00000100:00100000:1.0:1776188933.500412:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654934080 00000100:00100000:1.0:1776188933.500442:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1278 00000100:00100000:1.0:1776188933.500446:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cd00b680 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654934080:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776188933.500480:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cd00b680 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654934080:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 30us (427us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188933.500485:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1278 00000100:00100000:3.0:1776188933.500943:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654934208 00000100:00100000:0.0:1776188933.500946:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654934144 00000100:00100000:0.0:1776188933.500989:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9600 00000100:00100000:0.0:1776188933.500994:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003f45b220 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654934144:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776188933.501039:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003f45b220 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654934144:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 44us (1073us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188933.501046:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9600 00000100:00100000:0.0:1776188933.501059:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57103 00000100:00100000:0.0:1776188933.501064:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f600e06c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654934208:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776188933.501078:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776188933.501110:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f600e06c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654934208:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 45us (1200us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188933.501117:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57103 00010000:02020000:1.0:1776188933.726025:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776188933.820971:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000adb1e111 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664258624:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776188933.821048:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188933.821090:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776188933.821095:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e139d197 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664258560:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776188933.821121:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188933.821224:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664258624 00000100:00100000:0.0:1776188933.821274:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664258560 00000100:00100000:0.0:1776188933.821293:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6440 00000100:00100000:0.0:1776188933.821299:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cc904d7f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664258624:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776188933.821322:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6441 00000100:00100000:3.0:1776188933.821327:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0c7825a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664258560:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776188933.821334:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00002000:00000024:3.0:1776188933.821344:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776188933.821378:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cc904d7f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664258624:12345-0@lo:13:osp-pre-1-0.0 Request processed in 78us (330us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188933.821379:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0c7825a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664258560:12345-0@lo:13:osp-pre-0-0.0 Request processed in 52us (259us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188933.821385:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6440 00000100:00100000:2.0:1776188933.821622:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e139d197 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664258560:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776188933.821634:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000adb1e111 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664258624:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776188933.823680:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6441 00010000:02020000:1.0:1776188933.980303:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188934.141957:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188934.236375:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188934.492284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188934.748289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188934.844772:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e75ea3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664258688:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776188934.844834:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188934.845314:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664258688 00000100:00100000:3.0:1776188934.845366:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6442 00000100:00100000:3.0:1776188934.845372:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c1af713e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664258688:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776188934.845394:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776188934.845461:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e75ea3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664258688:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776188934.845479:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e139d197 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664258752:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776188934.845512:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c1af713e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664258688:12345-0@lo:13:osp-pre-0-1.0 Request processed in 139us (679us total) trans 0 rc 0/0 00000100:00080000:0.0:1776188934.845603:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776188934.845607:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776188934.845611:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776188934.846032:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9076 00000100:00080000:0.0:1776188934.846035:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776188934.846039:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776188934.846124:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000fa770ed8 x1862459664258944/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776188934.846156:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776188934.846159:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776188934.846161:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776188934.846164:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9076 00000100:00080000:0.0:1776188934.846166:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776188934.846169:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776188934.846200:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006ef3ce93 x1862459664259072/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776188934.846208:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776188934.846210:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776188934.846211:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776188934.846214:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9076 00000100:00080000:0.0:1776188934.846216:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776188934.846218:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776188934.846228:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002871fa24 x1862459664259136/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776188934.846271:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024c95b46 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664258816:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188934.846309:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188934.846327:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f92d37ee pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664259264:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188934.846354:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188934.846364:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6442 00000100:00100000:0.0:1776188934.846377:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1776188934.846384:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094ac8433 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664258880:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188934.846400:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664259264 00000100:00100000:0.0:1776188934.846414:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188934.846432:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bc860b89 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664259328:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188934.846451:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188934.846479:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664258880 00000100:00100000:2.0:1776188934.846889:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8261 00000100:00100000:2.0:1776188934.846896:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e01c7a33 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664259264:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188934.847424:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e01c7a33 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664259264:12345-0@lo:400:kworker/u8:0.0 Request processed in 526us (1069us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188934.847432:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8261 00000100:00100000:2.0:1776188934.847474:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f92d37ee pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664259264:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188934.847515:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664259328 00000100:00100000:2.0:1776188934.847546:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9601 00000100:00100000:2.0:1776188934.847550:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009c73aa7b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664259328:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188934.847593:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009c73aa7b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664259328:12345-0@lo:400:kworker/u8:0.0 Request processed in 42us (1141us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188934.847602:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9601 00000100:00100000:1.0:1776188934.847606:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188934.847627:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000036ddffa8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664259008:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188934.847639:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000025584b84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664259200:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188934.847677:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188934.847681:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188934.847697:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002bdcaf54 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664259392:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188934.847719:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fa770ed8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664258944:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776188934.847721:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188934.847751:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188934.847769:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006ef3ce93 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664259072:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776188934.847790:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188934.847804:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002871fa24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664259136:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776188934.847826:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188934.847868:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664259008 00000100:00100000:1.0:1776188934.847879:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664258816 00000100:00100000:0.0:1776188934.847888:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664258944 00000100:00100000:0.0:1776188934.847903:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664259072 00000100:00100000:1.0:1776188934.847923:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1279 00000100:00100000:1.0:1776188934.847929:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f6dc323d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664258816:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188934.847961:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f6dc323d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664258816:12345-0@lo:400:kworker/u8:0.0 Request processed in 32us (1652us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188934.847968:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1279 00000100:00100000:0.0:1776188934.848332:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9602 00000100:00100000:1.0:1776188934.848341:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9603 00000100:00100000:1.0:1776188934.848349:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009ac61361 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664259008:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188934.848391:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009ac61361 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664259008:12345-0@lo:400:kworker/u8:0.0 Request processed in 43us (715us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188934.848398:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9603 00000100:00100000:2.0:1776188934.848409:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000036ddffa8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664259008:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188934.848417:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57104 00000100:00100000:1.0:1776188934.848421:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000476e266d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664258944:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776188934.848434:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776188934.848440:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57105 00000100:00100000:2.0:1776188934.848445:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fb6b7b14 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664259072:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776188934.848454:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776188934.848480:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000775c4677 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664258880:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188934.849105:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664259200 00000100:00100000:3.0:1776188934.849133:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664259392 00000100:00100000:3.0:1776188934.849150:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8262 00000100:00100000:3.0:1776188934.849154:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c68f859e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664259200:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188934.849250:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fb6b7b14 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664259072:12345-0@lo:38:kworker/u8:0.0 Request processed in 803us (1459us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776188934.849258:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57105 00000100:00100000:2.0:1776188934.849294:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bc860b89 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664259328:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188934.849304:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024c95b46 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664258816:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188934.849358:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8263 00000100:00100000:2.0:1776188934.849362:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e5080bed pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664259392:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188934.849396:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e5080bed pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664259392:12345-0@lo:400:kworker/u8:0.0 Request processed in 33us (1677us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188934.849401:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8263 00000100:00100000:2.0:1776188934.849429:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002bdcaf54 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664259392:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188934.849571:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664259136 00000100:00100000:2.0:1776188934.849594:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9034 00000100:00100000:2.0:1776188934.849598:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000862ba2d5 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664259136:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776188934.849607:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776188934.849634:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000862ba2d5 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664259136:12345-0@lo:38:kworker/u8:0.0 Request processed in 35us (1808us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776188934.849651:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9034 00000100:00100000:2.0:1776188934.849676:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664258752 00000100:00100000:2.0:1776188934.849698:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6443 00000100:00100000:2.0:1776188934.849701:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000028933edf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664258752:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776188934.849714:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776188934.849757:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000028933edf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664258752:12345-0@lo:13:osp-pre-1-1.0 Request processed in 55us (2158us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188934.849763:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6443 00000100:00100000:2.0:1776188934.849817:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e139d197 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664258752:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776188934.853383:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000775c4677 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664258880:12345-0@lo:400:kworker/u8:0.0 Request processed in 5035us (6962us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188934.853400:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9602 00000100:00100000:2.0:1776188934.856908:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094ac8433 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664258880:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188934.858956:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000476e266d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664258944:12345-0@lo:38:kworker/u8:0.0 Request processed in 10524us (11196us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188934.858973:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57104 00000100:00100000:3.0:1776188934.858996:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c68f859e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664259200:12345-0@lo:400:kworker/u8:0.0 Request processed in 9838us (11315us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188934.859007:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8262 00000100:00100000:0.0:1776188934.859131:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000025584b84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664259200:0@lo:400:kworker/u8:0.0 00000100:00080000:2.0:1776188934.859230:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776188934.859236:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776188934.859240:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fa770ed8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664258944:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776188934.859252:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776188934.859255:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776188934.859257:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006ef3ce93 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664259072:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776188934.859267:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776188934.859270:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776188934.859272:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002871fa24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664259136:0@lo:38:kworker/u8:0.0 00010000:02020000:1.0:1776188935.005010:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188935.164390:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188935.260971:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188935.516385:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188935.772286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188936.029053:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188936.188386:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188936.285042:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188936.540355:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188936.796493:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188937.052300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188937.213508:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188937.308372:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188937.564284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188937.820386:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188938.076290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188938.236986:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188938.332294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188938.588325:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776188938.618603:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188938.618649:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188938.618686:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188938.618699:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654934336 00000100:00100000:3.0:1776188938.618731:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776188938.618786:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654934464 00000100:00100000:0.0:1776188938.618822:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654934528 00000100:00100000:2.0:1776188938.618841:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1280 00000100:00100000:1.0:1776188938.618842:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654934400 00000100:00100000:2.0:1776188938.618849:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000078287bc4 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654934336:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776188938.618855:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8264 00000100:00100000:0.0:1776188938.618859:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a1abf153 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654934528:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776188938.618877:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9604 00000100:00100000:1.0:1776188938.618884:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007a0ce2a1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654934400:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776188938.618917:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a1abf153 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654934528:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 56us (232us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188938.618929:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8264 00000100:00100000:3.0:1776188938.618937:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57106 00000100:00100000:3.0:1776188938.618943:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ccb5377d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654934464:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00000100:00100000:2.0:1776188938.618956:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000078287bc4 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654934336:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 107us (360us total) trans 0 rc 0/0 00010000:02000000:3.0:1776188938.618962:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776188938.618967:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1280 00000100:00100000:3.0:1776188938.619004:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ccb5377d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654934464:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 59us (354us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188938.619011:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57106 00000100:00100000:1.0:1776188938.619108:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007a0ce2a1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654934400:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 224us (378us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188938.619115:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9604 00010000:02020000:1.0:1776188938.844291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776188938.940450:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006f76f00e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664259456:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776188938.940518:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188938.940715:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664259456 00000100:00100000:1.0:1776188938.940725:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776188938.940737:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000025a2ff5a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664259520:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776188938.940760:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6444 00000100:00100000:0.0:1776188938.940777:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c89da359 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664259456:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776188938.940784:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:0.0:1776188938.940806:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776188938.940850:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c89da359 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664259456:12345-0@lo:13:osp-pre-0-0.0 Request processed in 73us (332us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188938.940856:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6444 00000100:00100000:3.0:1776188938.940951:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006f76f00e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664259456:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776188938.942724:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664259520 00000100:00100000:0.0:1776188938.942774:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6445 00000100:00100000:0.0:1776188938.942779:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d6ea221c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664259520:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776188938.942807:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776188938.942873:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d6ea221c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664259520:12345-0@lo:13:osp-pre-1-0.0 Request processed in 93us (2090us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188938.942879:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6445 00000100:00100000:1.0:1776188938.942968:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000025a2ff5a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664259520:0@lo:13:osp-pre-1-0.0 00010000:02020000:1.0:1776188939.100296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188939.260386:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188939.356302:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188939.612294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188939.869037:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188939.964406:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776188939.964428:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000026c12667 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664259584:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776188939.964501:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188939.964581:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664259584 00000100:00100000:1.0:1776188939.964602:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000025a2ff5a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664259648:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776188939.964621:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6446 00000100:00100000:3.0:1776188939.964628:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006f76f00e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664259584:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776188939.964639:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:3.0:1776188939.964666:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776188939.964720:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006f76f00e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664259584:12345-0@lo:13:osp-pre-1-1.0 Request processed in 91us (220us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188939.964729:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6446 00000100:00100000:0.0:1776188939.964733:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664259648 00000100:00100000:0.0:1776188939.964771:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6447 00000100:00100000:0.0:1776188939.964777:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000def45ccb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664259648:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776188939.964789:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776188939.964832:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000def45ccb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664259648:12345-0@lo:13:osp-pre-0-1.0 Request processed in 55us (194us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188939.964839:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6447 00000100:00100000:1.0:1776188939.964918:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000026c12667 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664259584:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776188939.964992:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000025a2ff5a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664259648:0@lo:13:osp-pre-0-1.0 00000100:00080000:2.0:1776188939.965293:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776188939.965298:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776188939.965302:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776188939.965307:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9081 00000100:00080000:2.0:1776188939.965310:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00080000:2.0:1776188939.965313:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776188939.965331:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000da51a899 x1862459664259840/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776188939.965351:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776188939.965354:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776188939.965355:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776188939.965358:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9081 00000100:00080000:2.0:1776188939.965361:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00080000:2.0:1776188939.965364:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776188939.965372:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000078cf1bd6 x1862459664259968/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776188939.965380:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776188939.965382:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776188939.965383:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776188939.965386:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9081 00000100:00080000:2.0:1776188939.965388:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 6s 00000100:00080000:2.0:1776188939.965390:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776188939.965401:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000e75ea3b x1862459664260032/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776188939.965440:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000da51a899 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664259840:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776188939.965476:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188939.965491:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000078cf1bd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664259968:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776188939.965511:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188939.965524:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e75ea3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664260032:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776188939.965546:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188939.965574:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664259840 00000100:00100000:2.0:1776188939.965597:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664259968 00000100:00100000:2.0:1776188939.965611:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57107 00000100:00100000:2.0:1776188939.965615:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000094ac8433 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664259840:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776188939.965629:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776188939.965661:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000094ac8433 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664259840:12345-0@lo:38:kworker/u8:1.0 Request processed in 45us (185us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776188939.965667:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57107 00000100:00100000:2.0:1776188939.965679:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57108 00000100:00100000:2.0:1776188939.965682:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e139d197 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664259968:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776188939.965689:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776188939.965710:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e139d197 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664259968:12345-0@lo:38:kworker/u8:1.0 Request processed in 27us (199us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776188939.965716:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57108 00000100:00100000:2.0:1776188939.965733:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664260032 00000100:00100000:2.0:1776188939.965763:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9035 00000100:00100000:2.0:1776188939.965767:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000024c95b46 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664260032:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776188939.965774:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776188939.965795:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000024c95b46 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664260032:12345-0@lo:38:kworker/u8:1.0 Request processed in 28us (249us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776188939.965801:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9035 00000100:00080000:2.0:1776188939.965840:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776188939.965843:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776188939.965846:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000da51a899 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664259840:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776188939.965852:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776188939.965854:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776188939.965856:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000078cf1bd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664259968:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776188939.965861:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776188939.965863:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776188939.965865:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e75ea3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664260032:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776188939.968843:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a17ea634 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664260096:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188939.968916:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188939.968971:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1776188939.968978:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000db2d4fd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664259712:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188939.969010:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188939.969031:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f5d3008d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664260160:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188939.969052:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188939.969090:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664259712 00000100:00100000:3.0:1776188939.969137:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1281 00000100:00100000:3.0:1776188939.969142:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003afac4ae pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664259712:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188939.969227:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003afac4ae pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664259712:12345-0@lo:400:kworker/u8:1.0 Request processed in 84us (217us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188939.969235:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1281 00000100:00100000:3.0:1776188939.969282:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000db2d4fd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664259712:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188939.969375:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000db5ec222 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664259776:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188939.969407:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188939.969432:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fa770ed8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664260224:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188939.969454:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188939.969473:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776188939.969479:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000101b4f14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664259904:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188939.969502:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188939.969528:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664259776 00000100:00100000:1.0:1776188939.969552:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664260224 00000100:00100000:1.0:1776188939.969569:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664259904 00000100:00100000:1.0:1776188939.969583:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9605 00000100:00100000:1.0:1776188939.969587:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b181f293 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664259776:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188939.969621:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b181f293 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664259776:12345-0@lo:400:kworker/u8:1.0 Request processed in 33us (214us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188939.969627:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9605 00000100:00100000:1.0:1776188939.969641:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9606 00000100:00100000:1.0:1776188939.969644:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005ecd069c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664260224:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188939.969669:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005ecd069c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664260224:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (215us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188939.969674:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9606 00000100:00100000:1.0:1776188939.969686:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9607 00000100:00100000:1.0:1776188939.969689:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000050c951fb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664259904:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188939.969711:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000050c951fb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664259904:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (209us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188939.969716:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9607 00000100:00100000:1.0:1776188939.969763:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000101b4f14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664259904:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188939.969773:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000db5ec222 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664259776:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188939.969781:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fa770ed8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664260224:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188939.969870:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664260096 00000100:00100000:0.0:1776188939.969905:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664260160 00000100:00100000:0.0:1776188939.969924:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8265 00000100:00100000:0.0:1776188939.969928:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d2b21909 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664260096:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188939.969963:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d2b21909 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664260096:12345-0@lo:400:kworker/u8:1.0 Request processed in 34us (1048us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188939.969970:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8265 00000100:00100000:0.0:1776188939.969981:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8266 00000100:00100000:0.0:1776188939.969984:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c0bb1b99 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664260160:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188939.970009:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c0bb1b99 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664260160:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (957us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188939.970014:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8266 00000100:00100000:3.0:1776188939.970099:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f5d3008d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664260160:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188939.970109:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a17ea634 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664260096:0@lo:400:kworker/u8:1.0 00010000:02020000:1.0:1776188940.124365:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188940.284784:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188940.380503:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188940.636298:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188940.892301:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188941.149889:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776188941.308335:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188941.404285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188941.660383:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188941.916282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188942.172935:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188942.333758:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188942.428289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188942.684273:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188942.940286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188943.196297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188943.356401:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188943.455490:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188943.708291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776188943.738995:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188943.739043:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188943.739083:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188943.739154:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188943.739282:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654934784 00000100:00100000:1.0:1776188943.739291:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654934656 00000100:00100000:1.0:1776188943.739344:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654934720 00000100:00100000:2.0:1776188943.739345:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8267 00000100:00100000:2.0:1776188943.739353:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c4620d7d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2278:x1862459654934784:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1776188943.739368:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9608 00000100:00100000:1.0:1776188943.739375:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b7fee202 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654934656:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1776188943.739430:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b7fee202 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654934656:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 55us (445us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188943.739440:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9608 00000100:00100000:0.0:1776188943.739459:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654934592 00000100:00100000:1.0:1776188943.739463:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57109 00000100:00100000:1.0:1776188943.739469:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000006dfa37f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654934720:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:1.0:1776188943.739484:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776188943.739509:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1282 00000100:00100000:0.0:1776188943.739514:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000921cb3a0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654934592:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1776188943.741071:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c4620d7d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2278:x1862459654934784:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 1710us (1980us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188943.741085:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8267 00000100:00100000:1.0:1776188943.747395:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000006dfa37f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654934720:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 7915us (8231us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188943.747411:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57109 00000100:00100000:0.0:1776188943.747467:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000921cb3a0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654934592:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 7869us (8341us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188943.747479:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1282 00010000:02020000:1.0:1776188943.965220:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188944.060407:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001f4e478a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664260288:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776188944.060466:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776188944.060480:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188944.060483:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009015c790 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664260352:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776188944.060522:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188944.060568:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664260288 00000100:00100000:1.0:1776188944.060571:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664260352 00000100:00100000:3.0:1776188944.060612:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6448 00000100:00100000:3.0:1776188944.060618:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002bdcaf54 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664260288:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776188944.060647:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776188944.061806:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002bdcaf54 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664260288:12345-0@lo:13:osp-pre-1-0.0 Request processed in 1177us (1318us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188944.061820:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001f4e478a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664260288:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776188944.061820:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6448 00000100:00100000:3.0:1776188944.062118:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6449 00000100:00100000:3.0:1776188944.062124:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000060a8344d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664260352:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776188944.062244:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776188944.062427:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000060a8344d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664260352:12345-0@lo:13:osp-pre-0-0.0 Request processed in 302us (1905us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188944.062434:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6449 00000100:00100000:0.0:1776188944.063469:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009015c790 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664260352:0@lo:13:osp-pre-0-0.0 00010000:02020000:1.0:1776188944.220973:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188944.383402:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188944.477336:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188944.732270:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188944.988286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776188945.084374:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001f4e478a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664260416:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776188945.084441:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188945.084457:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776188945.084472:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000026c12667 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664260480:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776188945.084494:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664260416 00000100:00100000:0.0:1776188945.084513:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188945.084549:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664260480 00000100:00100000:3.0:1776188945.084584:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6450 00000100:00100000:3.0:1776188945.084590:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005bd91e44 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664260416:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776188945.084600:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6451 00000100:00100000:1.0:1776188945.084607:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001b0eae72 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664260480:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776188945.084611:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00002000:00000024:1.0:1776188945.084646:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776188945.084671:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005bd91e44 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664260416:12345-0@lo:13:osp-pre-0-1.0 Request processed in 79us (231us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188945.084680:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6450 00000100:00100000:1.0:1776188945.084688:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001b0eae72 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664260480:12345-0@lo:13:osp-pre-1-1.0 Request processed in 80us (175us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188945.084696:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6451 00000100:00100000:3.0:1776188945.084741:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001f4e478a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664260416:0@lo:13:osp-pre-0-1.0 00000100:00080000:1.0:1776188945.085555:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776188945.085561:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776188945.085565:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776188945.085570:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9086 00000100:00080000:1.0:1776188945.085573:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776188945.085577:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776188945.085597:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001f4e478a x1862459664260672/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776188945.085636:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776188945.085640:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776188945.085641:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776188945.085645:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9086 00000100:00080000:1.0:1776188945.085647:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776188945.085650:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776188945.085659:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004490682c x1862459664260800/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776188945.085667:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776188945.085669:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776188945.085670:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776188945.085674:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9086 00000100:00080000:1.0:1776188945.085676:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776188945.085679:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776188945.085689:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b4a5ce06 x1862459664260864/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776188945.085733:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fa770ed8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664260736:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188945.085776:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188945.085809:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1776188945.085815:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000adb1e111 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664260608:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188945.085838:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188945.085851:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000576bf6bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664261056:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188945.085871:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188945.085896:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664260736 00000100:00100000:1.0:1776188945.085926:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664260608 00000100:00100000:1.0:1776188945.085943:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664261056 00000100:00100000:1.0:1776188945.085958:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9609 00000100:00100000:1.0:1776188945.085962:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bd5316a5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664260736:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188945.086001:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bd5316a5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664260736:12345-0@lo:400:kworker/u8:0.0 Request processed in 37us (226us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188945.086020:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9609 00000100:00100000:1.0:1776188945.086036:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9610 00000100:00100000:1.0:1776188945.086039:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001a975e34 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664260608:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188945.086066:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001a975e34 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664260608:12345-0@lo:400:kworker/u8:0.0 Request processed in 26us (228us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188945.086072:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9610 00000100:00100000:1.0:1776188945.086084:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9611 00000100:00100000:1.0:1776188945.086087:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005ee0b9e5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664261056:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188945.086115:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005ee0b9e5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664261056:12345-0@lo:400:kworker/u8:0.0 Request processed in 26us (243us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188945.086120:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9611 00000100:00100000:1.0:1776188945.086158:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000adb1e111 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664260608:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188945.086168:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000576bf6bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664261056:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188945.086200:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fa770ed8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664260736:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188945.086360:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001f4e478a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664260672:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776188945.086392:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188945.086412:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004490682c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664260800:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776188945.086436:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188945.086451:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4a5ce06 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664260864:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776188945.086474:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188945.086503:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664260672 00000100:00100000:2.0:1776188945.086522:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664260800 00000100:00100000:2.0:1776188945.086533:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57110 00000100:00100000:2.0:1776188945.086536:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a17ea634 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664260672:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776188945.086549:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776188945.086582:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a17ea634 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664260672:12345-0@lo:38:kworker/u8:0.0 Request processed in 45us (191us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776188945.086589:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57110 00000100:00100000:2.0:1776188945.086600:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57111 00000100:00100000:2.0:1776188945.086603:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f5d3008d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664260800:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776188945.086609:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776188945.086655:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f5d3008d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664260800:12345-0@lo:38:kworker/u8:0.0 Request processed in 50us (219us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776188945.086660:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57111 00000100:00100000:2.0:1776188945.086678:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664260864 00000100:00100000:2.0:1776188945.086695:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9036 00000100:00100000:2.0:1776188945.086699:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000db5ec222 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664260864:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776188945.086705:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776188945.086727:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000db5ec222 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664260864:12345-0@lo:38:kworker/u8:0.0 Request processed in 27us (253us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776188945.086732:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9036 00000100:00100000:2.0:1776188945.086754:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5f1e324 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664260928:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188945.086777:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188945.086794:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1776188945.086800:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000025a2ff5a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664260544:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188945.086824:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188945.086972:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664260928 00000100:00100000:1.0:1776188945.087010:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8268 00000100:00100000:1.0:1776188945.087014:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000101b4f14 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664260928:12345-0@lo:400:kworker/u8:0.0 00000100:00080000:3.0:1776188945.087018:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188945.087024:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188945.087029:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001f4e478a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664260672:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776188945.087041:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188945.087044:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188945.087047:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004490682c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664260800:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776188945.087054:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000101b4f14 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664260928:12345-0@lo:400:kworker/u8:0.0 Request processed in 37us (275us total) trans 0 rc 0/0 00000100:00080000:3.0:1776188945.087059:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:1.0:1776188945.087062:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8268 00000100:00080000:3.0:1776188945.087065:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188945.087070:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4a5ce06 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664260864:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776188945.088416:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000026c12667 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664260480:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776188945.088561:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000549bc579 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664260992:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188945.088606:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188945.088655:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5f1e324 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664260928:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188945.088820:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664260992 00000100:00100000:1.0:1776188945.088863:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8269 00000100:00100000:1.0:1776188945.088867:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000db2d4fd6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664260992:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188945.089103:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000db2d4fd6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664260992:12345-0@lo:400:kworker/u8:0.0 Request processed in 235us (498us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188945.089112:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8269 00000100:00100000:0.0:1776188945.089248:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664260544 00000100:00100000:0.0:1776188945.089487:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1283 00000100:00100000:0.0:1776188945.089493:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008da73e5e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664260544:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188945.089529:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008da73e5e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664260544:12345-0@lo:400:kworker/u8:0.0 Request processed in 36us (2705us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188945.089535:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1283 00000100:00100000:2.0:1776188945.089626:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000549bc579 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664260992:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188945.089656:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000025a2ff5a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664260544:0@lo:400:kworker/u8:0.0 00010000:02020000:1.0:1776188945.246130:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776188945.404344:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188945.500290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188945.756304:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188946.012782:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188946.268300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188946.428576:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188946.524305:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188946.780289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188947.036323:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188947.294072:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188947.452706:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188947.548293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188947.804245:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188948.063751:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188948.317224:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188948.476692:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188948.572472:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188948.828430:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776188948.858886:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188948.858930:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188948.858964:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188948.859040:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776188948.859092:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654934912 00000100:00100000:1.0:1776188948.859144:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654934976 00000100:00100000:1.0:1776188948.859165:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9612 00000100:00100000:1.0:1776188948.859195:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000e75ea3b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654934912:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1776188948.859267:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000e75ea3b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654934912:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 95us (337us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188948.859274:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9612 00000100:00100000:1.0:1776188948.859289:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57112 00000100:00100000:1.0:1776188948.859293:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000078cf1bd6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654934976:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:1.0:1776188948.859307:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776188948.859327:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000078cf1bd6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654934976:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 34us (364us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188948.859332:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57112 00000100:00100000:1.0:1776188948.859349:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654935040 00000100:00100000:1.0:1776188948.859368:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8270 00000100:00100000:1.0:1776188948.859371:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000da51a899 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2279:x1862459654935040:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1776188948.859391:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000da51a899 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2279:x1862459654935040:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 20us (352us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188948.859398:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8270 00000100:00100000:0.0:1776188948.861640:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654934848 00000100:00100000:0.0:1776188948.861713:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1284 00000100:00100000:0.0:1776188948.861720:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c3d39af pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654934848:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776188948.861808:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c3d39af pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654934848:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 88us (2929us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188948.861817:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1284 00010000:02020000:1.0:1776188949.084322:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776188949.180686:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776188949.180707:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000023a5b7b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664261120:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776188949.180772:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188949.180817:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664261120 00000100:00100000:1.0:1776188949.180862:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6452 00000100:00100000:1.0:1776188949.180868:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c8b239a6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664261120:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776188949.180895:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776188949.180935:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c8b239a6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664261120:12345-0@lo:13:osp-pre-1-0.0 Request processed in 66us (163us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188949.180941:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6452 00000100:00100000:1.0:1776188949.180986:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000023a5b7b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664261120:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776188949.182014:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776188949.182031:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6665a35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664261184:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776188949.182069:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188949.182155:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664261184 00000100:00100000:1.0:1776188949.182224:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6453 00000100:00100000:1.0:1776188949.182229:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003b5d789a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664261184:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776188949.182264:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776188949.182303:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003b5d789a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664261184:12345-0@lo:13:osp-pre-0-0.0 Request processed in 73us (234us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188949.182311:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6453 00000100:00100000:3.0:1776188949.182878:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6665a35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664261184:0@lo:13:osp-pre-0-0.0 00010000:02020000:1.0:1776188949.340494:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776188949.500502:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188949.596295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188949.852302:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188950.109605:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776188950.204371:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776188950.204393:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002ef57fef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664261248:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776188950.204460:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188950.204604:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664261248 00000100:00100000:1.0:1776188950.204654:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6454 00000100:00100000:1.0:1776188950.204659:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002871fa24 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664261248:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776188950.204690:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776188950.204733:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002871fa24 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664261248:12345-0@lo:13:osp-pre-1-1.0 Request processed in 72us (273us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188950.204740:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6454 00000100:00100000:3.0:1776188950.204796:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000023a5b7b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664261312:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776188950.204830:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188950.204857:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776188950.204866:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006ef3ce93 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664261376:0@lo:400:kworker/u8:1.0 00000100:00080000:2.0:1776188950.204870:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776188950.204876:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776188950.204879:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776188950.204884:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9092 00000100:00080000:2.0:1776188950.204887:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:2.0:1776188950.204892:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188950.204898:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188950.204909:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bc860b89 x1862459664261504/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776188950.204930:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664261376 00000100:00080000:2.0:1776188950.204935:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776188950.204939:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776188950.204940:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776188950.204944:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9092 00000100:00080000:2.0:1776188950.204946:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00080000:2.0:1776188950.204949:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776188950.204959:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f92d37ee x1862459664261632/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776188950.204966:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1285 00000100:00080000:2.0:1776188950.204968:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776188950.204973:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776188950.204974:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776188950.204977:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9092 00000100:00080000:2.0:1776188950.204987:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00080000:2.0:1776188950.204989:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776188950.205000:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b3b3d75f x1862459664261696/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776188950.205041:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000036ddffa8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664261568:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776188950.205068:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188950.205084:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1776188950.205088:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000025584b84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664261440:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776188950.205109:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188950.205121:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ce3ffae5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664261888:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776188950.205139:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188950.205193:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000054387670 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664261824:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188950.205193:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e858e44a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664261376:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776188950.205230:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188950.205253:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776188950.205259:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e858e44a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664261376:12345-0@lo:400:kworker/u8:1.0 Request processed in 284us (360us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188950.205261:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000db568034 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664261760:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188950.205283:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1285 00000100:00100000:2.0:1776188950.205291:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188950.205318:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664261824 00000100:00100000:3.0:1776188950.205322:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bc860b89 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664261504:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776188950.205343:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664261760 00000100:00100000:2.0:1776188950.205360:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8271 00000100:00100000:2.0:1776188950.205369:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004ef5af0c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664261824:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188950.205370:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188950.205389:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f92d37ee pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664261632:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776188950.205412:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004ef5af0c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664261824:12345-0@lo:400:kworker/u8:1.0 Request processed in 46us (181us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188950.205422:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8271 00000100:00100000:3.0:1776188950.205426:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188950.205438:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8272 00000100:00100000:2.0:1776188950.205443:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b3e4a548 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664261760:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188950.205451:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3b3d75f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664261696:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776188950.205480:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b3e4a548 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664261760:12345-0@lo:400:kworker/u8:1.0 Request processed in 36us (191us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188950.205490:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8272 00000100:00100000:3.0:1776188950.205490:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188950.205514:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664261568 00000100:00100000:3.0:1776188950.205546:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664261440 00000100:00100000:2.0:1776188950.205556:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664261888 00000100:00100000:3.0:1776188950.205573:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664261504 00000100:00100000:2.0:1776188950.205577:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664261632 00000100:00100000:2.0:1776188950.205594:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9613 00000100:00100000:2.0:1776188950.205597:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000545eaa39 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664261568:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188950.205605:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9614 00000100:00100000:3.0:1776188950.205611:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000043ca7548 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664261440:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776188950.205630:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000545eaa39 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664261568:12345-0@lo:400:kworker/u8:1.0 Request processed in 31us (561us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188950.205636:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9613 00000100:00100000:2.0:1776188950.205651:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9615 00000100:00100000:2.0:1776188950.205655:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006c90bb74 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664261888:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188950.205660:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000043ca7548 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664261440:12345-0@lo:400:kworker/u8:1.0 Request processed in 47us (549us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188950.205668:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9614 00000100:00100000:3.0:1776188950.205687:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57113 00000100:00100000:2.0:1776188950.205695:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006c90bb74 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664261888:12345-0@lo:400:kworker/u8:1.0 Request processed in 36us (551us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188950.205696:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009015c790 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664261504:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776188950.205708:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9615 00010000:02000000:3.0:1776188950.205715:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776188950.205723:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57114 00000100:00100000:2.0:1776188950.205727:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000021503dd8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664261632:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776188950.205736:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188950.205772:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009015c790 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664261504:12345-0@lo:38:kworker/u8:1.0 Request processed in 79us (408us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188950.205778:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57113 00000100:00100000:3.0:1776188950.205796:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664261696 00000100:00100000:3.0:1776188950.205815:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9037 00000100:00100000:3.0:1776188950.205818:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000071348c99 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664261696:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776188950.205825:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776188950.206094:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000021503dd8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664261632:12345-0@lo:38:kworker/u8:1.0 Request processed in 364us (673us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776188950.206100:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57114 00000100:00100000:2.0:1776188950.206141:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000025584b84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664261440:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776188950.206152:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ce3ffae5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664261888:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776188950.206191:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000036ddffa8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664261568:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776188950.206225:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000db568034 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664261760:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776188950.206233:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000054387670 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664261824:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776188950.206265:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002ef57fef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664261248:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776188950.206303:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006ef3ce93 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664261376:0@lo:400:kworker/u8:1.0 00000100:00080000:2.0:1776188950.206407:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776188950.206411:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776188950.206414:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bc860b89 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664261504:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776188950.206422:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776188950.206424:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776188950.206427:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f92d37ee pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664261632:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776188950.206493:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664261312 00000100:00100000:0.0:1776188950.206530:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6455 00000100:00100000:0.0:1776188950.206534:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e6665a35 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664261312:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776188950.206548:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776188950.206587:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e6665a35 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664261312:12345-0@lo:13:osp-pre-0-1.0 Request processed in 52us (1759us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188950.206593:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6455 00000100:00100000:0.0:1776188950.206643:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000023a5b7b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664261312:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776188950.207709:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000071348c99 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664261696:12345-0@lo:38:kworker/u8:1.0 Request processed in 1816us (2150us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188950.207720:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9037 00000100:00080000:2.0:1776188950.208157:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776188950.208414:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776188950.208419:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3b3d75f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664261696:0@lo:38:kworker/u8:1.0 00010000:02020000:1.0:1776188950.365056:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188950.525578:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188950.620316:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188950.876370:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188951.132312:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188951.388787:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188951.548699:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188951.646712:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188951.900332:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188952.157971:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188952.412235:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188952.572448:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188952.668271:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188952.924303:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188953.180266:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188953.436277:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776188953.596723:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188953.692281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188953.948294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776188953.978316:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188953.979826:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654935232 00000100:00100000:3.0:1776188953.979888:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57115 00000100:00100000:3.0:1776188953.979894:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000054387670 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654935232:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:3.0:1776188953.979913:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776188953.979988:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000054387670 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654935232:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 93us (1682us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188953.979994:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57115 00000100:00100000:3.0:1776188953.981018:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188953.981139:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188953.981163:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188953.981221:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654935168 00000100:00100000:3.0:1776188953.981308:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9616 00000100:00100000:3.0:1776188953.981314:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ab3412dc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654935168:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776188953.981372:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ab3412dc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654935168:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 58us (358us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188953.981380:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9616 00000100:00100000:0.0:1776188953.981939:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654935104 00000100:00100000:0.0:1776188953.981975:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1286 00000100:00100000:0.0:1776188953.981980:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ff8d0a76 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654935104:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776188953.981982:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654935296 00000100:00100000:0.0:1776188953.982012:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ff8d0a76 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654935104:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 32us (875us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188953.982017:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1286 00000100:00100000:2.0:1776188953.982031:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8273 00000100:00100000:2.0:1776188953.982037:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c3185a52 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654935296:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776188953.982068:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c3185a52 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654935296:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 31us (905us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188953.982086:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8273 00010000:02020000:1.0:1776188954.204293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776188954.300468:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776188954.300490:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004ac5ba05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664262016:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776188954.300730:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188954.300819:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776188954.300831:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000013c0eaff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664261952:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776188954.300939:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188954.306556:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664262016 00000100:00100000:3.0:1776188954.306622:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664261952 00000100:00100000:3.0:1776188954.306656:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6456 00000100:00100000:3.0:1776188954.306662:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000032f89d8f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664262016:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776188954.306692:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776188954.306766:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000032f89d8f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664262016:12345-0@lo:13:osp-pre-1-0.0 Request processed in 102us (6214us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188954.306773:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6456 00000100:00100000:3.0:1776188954.306790:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6457 00000100:00100000:3.0:1776188954.306794:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008b54d6c4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664261952:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776188954.306809:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776188954.306837:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008b54d6c4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664261952:12345-0@lo:13:osp-pre-0-0.0 Request processed in 42us (5900us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188954.306843:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6457 00000100:00100000:3.0:1776188954.306886:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004ac5ba05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664262016:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776188954.306975:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000013c0eaff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664261952:0@lo:13:osp-pre-0-0.0 00010000:02020000:1.0:1776188954.460291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188954.620363:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188954.716289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188954.972539:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188955.229759:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776188955.324586:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ce3ffae5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664262144:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188955.324590:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f34aeb18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664262080:0@lo:13:osp-pre-0-1.0 00000100:00080000:2.0:1776188955.324676:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776188955.324680:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776188955.324683:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776188955.324688:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9097 00000100:00080000:2.0:1776188955.324691:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776188955.324696:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776188955.324712:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000029c2d1d5 x1862459664262272/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776188955.324732:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776188955.324735:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776188955.324737:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776188955.324740:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9097 00000100:00080000:2.0:1776188955.324741:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776188955.324744:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776188955.324753:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b3b3d75f x1862459664262400/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776188955.324760:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776188955.324763:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776188955.324764:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776188955.324767:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9097 00000100:00080000:2.0:1776188955.324769:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776188955.324771:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776188955.324794:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000db568034 x1862459664262464/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776188955.324841:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002ac4dbc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664262208:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188955.325015:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa4e2605 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664262336:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188955.325080:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188955.325117:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f92d37ee pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664262528:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188955.325141:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188955.325162:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000025584b84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664262784:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188955.325204:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188955.325238:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664262528 00000100:00100000:3.0:1776188955.325278:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664262784 00000100:00100000:3.0:1776188955.325300:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8274 00000100:00100000:3.0:1776188955.325304:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000013c0eaff pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664262528:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188955.325351:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000013c0eaff pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664262528:12345-0@lo:400:kworker/u8:1.0 Request processed in 46us (209us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188955.325357:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8274 00000100:00100000:3.0:1776188955.325375:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8275 00000100:00100000:3.0:1776188955.325379:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000088c2a62e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664262784:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188955.325428:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000088c2a62e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664262784:12345-0@lo:400:kworker/u8:1.0 Request processed in 48us (223us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188955.325434:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8275 00000100:00100000:3.0:1776188955.325457:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664262080 00000100:00100000:3.0:1776188955.325482:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6458 00000100:00100000:3.0:1776188955.325486:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000066892405 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664262080:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776188955.325499:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776188955.325529:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000066892405 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664262080:12345-0@lo:13:osp-pre-0-1.0 Request processed in 42us (449us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188955.325535:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6458 00000100:00100000:1.0:1776188955.325563:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188955.325580:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f92d37ee pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664262528:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188955.325597:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000025584b84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664262784:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188955.325600:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664262144 00000100:00100000:3.0:1776188955.325614:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f34aeb18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664262080:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776188955.325636:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1287 00000100:00100000:1.0:1776188955.325641:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f07038bd pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664262144:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188955.325662:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000029c2d1d5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664262272:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776188955.325681:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f07038bd pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664262144:12345-0@lo:400:kworker/u8:1.0 Request processed in 38us (119us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188955.325689:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1287 00000100:00100000:3.0:1776188955.325698:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188955.325722:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3b3d75f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664262400:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776188955.325726:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bc860b89 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664262592:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188955.325751:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188955.325763:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188955.325771:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000db568034 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664262464:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776188955.325789:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664262592 00000100:00100000:3.0:1776188955.325802:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188955.325821:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8276 00000100:00100000:1.0:1776188955.325826:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f34aeb18 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664262592:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188955.325840:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664262272 00000100:00100000:1.0:1776188955.325864:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f34aeb18 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664262592:12345-0@lo:400:kworker/u8:1.0 Request processed in 36us (101us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188955.325866:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664262400 00000100:00100000:1.0:1776188955.325876:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8276 00000100:00100000:3.0:1776188955.325886:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57116 00000100:00100000:3.0:1776188955.325891:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f435eca pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664262272:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776188955.325901:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57117 00010000:02000000:3.0:1776188955.325905:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776188955.325908:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005ad21a28 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664262400:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776188955.325933:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188955.325945:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f435eca pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664262272:12345-0@lo:38:kworker/u8:1.0 Request processed in 52us (247us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188955.325952:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57116 00000100:00100000:1.0:1776188955.325970:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005ad21a28 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664262400:12345-0@lo:38:kworker/u8:1.0 Request processed in 62us (219us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188955.325975:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664262464 00000100:00100000:1.0:1776188955.325978:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57117 00000100:00100000:3.0:1776188955.325998:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9038 00000100:00100000:1.0:1776188955.326005:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002ef57fef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664262848:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776188955.326005:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd5e571c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664262464:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776188955.326017:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776188955.326037:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188955.326052:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd5e571c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664262464:12345-0@lo:38:kworker/u8:1.0 Request processed in 47us (250us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188955.326058:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9038 00000100:00080000:3.0:1776188955.326085:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188955.326089:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188955.326092:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000029c2d1d5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664262272:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776188955.326099:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188955.326101:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188955.326103:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3b3d75f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664262400:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776188955.326110:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188955.326112:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188955.326114:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000db568034 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664262464:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776188955.326444:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664262848 00000100:00100000:1.0:1776188955.326477:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6459 00000100:00100000:1.0:1776188955.326480:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004ac5ba05 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664262848:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776188955.326494:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776188955.326632:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004ac5ba05 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664262848:12345-0@lo:13:osp-pre-1-1.0 Request processed in 150us (597us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188955.326640:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6459 00000100:00100000:1.0:1776188955.326666:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ce3ffae5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664262144:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188955.326685:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bc860b89 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664262592:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188955.326697:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002ef57fef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664262848:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776188955.326749:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188955.326775:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000036ddffa8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664262720:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188955.326798:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188955.326898:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664262720 00000100:00100000:1.0:1776188955.326931:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8277 00000100:00100000:1.0:1776188955.326934:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000026c12667 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664262720:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188955.326976:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000026c12667 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664262720:12345-0@lo:400:kworker/u8:1.0 Request processed in 41us (178us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188955.326983:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8277 00000100:00100000:1.0:1776188955.327007:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664262336 00000100:00100000:2.0:1776188955.327022:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000036ddffa8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664262720:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188955.327041:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9617 00000100:00100000:3.0:1776188955.327289:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188955.327321:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006ef3ce93 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664262656:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188955.327346:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188955.327376:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664262208 00000100:00100000:1.0:1776188955.327388:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002ef57fef pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664262336:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188955.327441:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002ef57fef pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664262336:12345-0@lo:400:kworker/u8:1.0 Request processed in 55us (693us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188955.327445:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664262656 00000100:00100000:3.0:1776188955.330528:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9618 00000100:00100000:3.0:1776188955.330539:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000db568034 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664262208:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188955.330652:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000db568034 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664262208:12345-0@lo:400:kworker/u8:1.0 Request processed in 115us (3364us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188955.330661:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9618 00000100:00100000:3.0:1776188955.330721:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002ac4dbc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664262208:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188955.330848:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa4e2605 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664262336:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188955.335449:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9617 00000100:00100000:3.0:1776188955.336645:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9619 00000100:00100000:3.0:1776188955.336656:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b3b3d75f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664262656:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188955.336758:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b3b3d75f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664262656:12345-0@lo:400:kworker/u8:1.0 Request processed in 103us (9411us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188955.336765:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9619 00000100:00100000:0.0:1776188955.336925:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006ef3ce93 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664262656:0@lo:400:kworker/u8:1.0 00010000:02020000:1.0:1776188955.484476:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188955.646609:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188955.742021:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188955.996789:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188956.252265:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188956.510317:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188956.668398:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188956.769086:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188957.020295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188957.276284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188957.532316:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188957.692413:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188957.788419:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188958.044593:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188958.300289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188958.561469:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188958.717077:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188958.812282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188959.069836:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776188959.098580:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188959.098671:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654935360 00000100:00100000:0.0:1776188959.099010:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1288 00000100:00100000:0.0:1776188959.099018:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005039a00d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654935360:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776188959.099116:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005039a00d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654935360:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 97us (543us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188959.099123:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1288 00000100:00100000:1.0:1776188959.112029:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776188959.112088:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188959.114694:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654935424 00000100:00100000:2.0:1776188959.114788:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654935488 00000100:00100000:2.0:1776188959.114989:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9620 00000100:00100000:2.0:1776188959.114998:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000023a5b7b3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654935424:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1776188959.115123:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000023a5b7b3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654935424:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 124us (3107us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188959.115133:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9620 00000100:00100000:2.0:1776188959.115160:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57118 00000100:00100000:2.0:1776188959.115164:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fa770ed8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654935488:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:2.0:1776188959.115216:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776188959.115247:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fa770ed8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654935488:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 82us (3160us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776188959.115255:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57118 00000100:00100000:2.0:1776188959.122911:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188959.123093:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654935552 00000100:00100000:0.0:1776188959.123221:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8278 00000100:00100000:0.0:1776188959.123231:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000036ddffa8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2280:x1862459654935552:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776188959.123349:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000036ddffa8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2280:x1862459654935552:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 117us (449us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188959.123362:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8278 00010000:02020000:1.0:1776188959.324352:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188959.425382:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000549bc579 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664262912:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776188959.425531:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188959.425791:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664262912 00000100:00100000:1.0:1776188959.425885:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6460 00000100:00100000:1.0:1776188959.425892:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008655ce15 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664262912:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776188959.425946:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776188959.426020:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008655ce15 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664262912:12345-0@lo:13:osp-pre-1-0.0 Request processed in 126us (493us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188959.426028:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6460 00000100:00100000:2.0:1776188959.426225:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000549bc579 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664262912:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776188959.426687:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776188959.426703:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa4e2605 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664262976:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776188959.426747:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188959.427591:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664262976 00000100:00100000:3.0:1776188959.427664:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6461 00000100:00100000:3.0:1776188959.427670:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008e777483 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664262976:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776188959.427706:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776188959.427783:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008e777483 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664262976:12345-0@lo:13:osp-pre-0-0.0 Request processed in 112us (1036us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188959.427791:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6461 00000100:00100000:0.0:1776188959.427919:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa4e2605 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664262976:0@lo:13:osp-pre-0-0.0 00010000:02020000:1.0:1776188959.580358:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188959.740482:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188959.839669:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188960.093613:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188960.348342:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776188960.444995:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000071b1847 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664263040:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776188960.445097:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1776188960.445278:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188960.445285:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188960.445289:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188960.445296:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9102 00000100:00080000:3.0:1776188960.445308:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776188960.445315:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188960.445338:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000029c2d1d5 x1862459664263232/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776188960.445504:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664263040 00000100:00100000:2.0:1776188960.445561:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6462 00000100:00100000:2.0:1776188960.445580:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b5f1e324 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664263040:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776188960.445608:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776188960.447870:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003bb8357b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664263360:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776188960.447989:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1776188960.448136:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188960.448145:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188960.448149:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188960.448155:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9102 00000100:00080000:3.0:1776188960.448159:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776188960.448166:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188960.448223:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009fddac3a x1862459664263424/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776188960.448241:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776188960.448244:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776188960.448245:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776188960.448249:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9102 00000100:00080000:3.0:1776188960.448251:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776188960.448254:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188960.448265:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000051b336a8 x1862459664263488/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776188960.448343:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000029c2d1d5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664263232:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188960.448397:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188960.448420:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009fddac3a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664263424:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188960.448456:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188960.448481:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000051b336a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664263488:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188960.448507:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188960.448561:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664263360 00000100:00100000:3.0:1776188960.448621:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6463 00000100:00100000:3.0:1776188960.448627:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b4a5ce06 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664263360:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776188960.448657:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776188960.448697:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b4a5ce06 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664263360:12345-0@lo:13:osp-pre-0-1.0 Request processed in 70us (710us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188960.448704:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6463 00000100:00100000:3.0:1776188960.448733:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664263488 00000100:00100000:3.0:1776188960.448754:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9039 00000100:00100000:3.0:1776188960.448758:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bc3d254c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664263488:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776188960.448775:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188960.448809:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bc3d254c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664263488:12345-0@lo:38:kworker/u8:0.0 Request processed in 50us (301us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188960.448816:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9039 00000100:00080000:3.0:1776188960.448856:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188960.448860:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188960.448865:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000051b336a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664263488:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188960.448893:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa4e2605 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664263296:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188960.448920:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188960.448940:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005ab67562 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664263680:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188960.448961:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188960.448994:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002ac4dbc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664263104:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188960.449024:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188960.449039:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f61bb34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664263552:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188960.449064:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188960.449081:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1776188960.449087:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ce3ffae5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664263168:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188960.449109:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188960.449124:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e071bb1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664263616:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188960.449146:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188960.449208:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664263232 00000100:00100000:3.0:1776188960.449231:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664263424 00000100:00100000:3.0:1776188960.449238:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664263296 00000100:00100000:3.0:1776188960.449256:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664263680 00000100:00100000:3.0:1776188960.449265:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664263168 00000100:00100000:3.0:1776188960.449277:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9621 00000100:00100000:3.0:1776188960.449282:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c5a99ce9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664263296:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188960.449336:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c5a99ce9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664263296:12345-0@lo:400:kworker/u8:0.0 Request processed in 54us (416us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188960.449343:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9621 00000100:00100000:3.0:1776188960.449356:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9622 00000100:00100000:3.0:1776188960.449359:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e3a29c53 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664263680:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188960.449383:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e3a29c53 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664263680:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (422us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188960.449388:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9622 00000100:00100000:3.0:1776188960.449399:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9623 00000100:00100000:3.0:1776188960.449402:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000051b336a8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664263168:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188960.449430:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000051b336a8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664263168:12345-0@lo:400:kworker/u8:0.0 Request processed in 26us (320us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188960.449435:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9623 00000100:00100000:3.0:1776188960.449459:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57119 00000100:00100000:3.0:1776188960.449463:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a6fd34e4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664263232:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776188960.449472:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188960.449507:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a6fd34e4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664263232:12345-0@lo:38:kworker/u8:0.0 Request processed in 44us (1111us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188960.449513:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57119 00000100:00100000:3.0:1776188960.449524:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57120 00000100:00100000:3.0:1776188960.449527:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001e4997ad pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664263424:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776188960.449534:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188960.449556:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001e4997ad pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664263424:12345-0@lo:38:kworker/u8:0.0 Request processed in 27us (1100us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188960.449561:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57120 00000100:00080000:3.0:1776188960.449592:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188960.449595:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188960.449599:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000029c2d1d5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664263232:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776188960.449607:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188960.449610:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188960.449612:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009fddac3a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664263424:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188960.449657:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa4e2605 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664263296:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188960.449667:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005ab67562 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664263680:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188960.449689:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ce3ffae5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664263168:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188960.449776:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664263552 00000100:00100000:3.0:1776188960.449802:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664263616 00000100:00100000:3.0:1776188960.449816:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8279 00000100:00100000:3.0:1776188960.449819:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006cdcc0ce pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664263552:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188960.449853:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006cdcc0ce pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664263552:12345-0@lo:400:kworker/u8:0.0 Request processed in 33us (789us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188960.449860:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8279 00000100:00100000:3.0:1776188960.449871:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8280 00000100:00100000:3.0:1776188960.449874:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002e574ef8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664263616:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188960.449898:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002e574ef8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664263616:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (752us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188960.449904:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8280 00000100:00100000:3.0:1776188960.449923:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664263104 00000100:00100000:3.0:1776188960.449949:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1289 00000100:00100000:3.0:1776188960.449953:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000080a5522a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664263104:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188960.449981:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000080a5522a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664263104:12345-0@lo:400:kworker/u8:0.0 Request processed in 26us (956us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188960.449987:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1289 00000100:00100000:3.0:1776188960.450032:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e071bb1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664263616:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188960.450041:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002ac4dbc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664263104:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188960.450050:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f61bb34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664263552:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188960.462864:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b5f1e324 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664263040:12345-0@lo:13:osp-pre-1-1.0 Request processed in 17280us (17753us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188960.462886:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6462 00000100:00100000:1.0:1776188960.463291:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000071b1847 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664263040:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776188960.463353:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003bb8357b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664263360:0@lo:13:osp-pre-0-1.0 00010000:02020000:1.0:1776188960.605142:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188960.765552:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188960.860338:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188961.117907:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188961.372357:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188961.628329:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188961.789283:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188961.884356:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188962.140350:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188962.396411:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188962.652405:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188962.815923:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188962.909389:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188963.164643:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188963.420388:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188963.676415:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188963.840335:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776188963.932517:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776188964.188608:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776188964.222039:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188964.222086:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188964.222115:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188964.222295:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188964.222330:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654935808 00000100:00100000:2.0:1776188964.222375:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8281 00000100:00100000:2.0:1776188964.222381:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000025584b84 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654935808:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1776188964.222447:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000025584b84 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654935808:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 65us (152us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188964.222454:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8281 00000100:00100000:3.0:1776188964.222691:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654935616 00000100:00100000:3.0:1776188964.222725:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1290 00000100:00100000:3.0:1776188964.222729:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f99b7a85 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654935616:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1776188964.222764:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f99b7a85 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654935616:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 33us (732us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188964.222770:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1290 00000100:00100000:1.0:1776188964.223493:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654935680 00000100:00100000:1.0:1776188964.223564:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654935744 00000100:00100000:1.0:1776188964.223589:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9624 00000100:00100000:1.0:1776188964.223594:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006ef3ce93 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654935680:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1776188964.223634:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006ef3ce93 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654935680:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 39us (1547us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188964.223640:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9624 00000100:00100000:1.0:1776188964.223654:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57121 00000100:00100000:1.0:1776188964.223658:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bc860b89 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654935744:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:1.0:1776188964.223672:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776188964.223696:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bc860b89 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654935744:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 37us (1580us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188964.223702:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57121 00010000:02020000:0.0:1776188964.444407:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188964.542005:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034c47c21 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664263744:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776188964.542072:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188964.542166:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664263744 00000100:00100000:3.0:1776188964.542249:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6464 00000100:00100000:3.0:1776188964.542255:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022af59eb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664263744:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776188964.542281:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776188964.542322:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022af59eb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664263744:12345-0@lo:13:osp-pre-0-0.0 Request processed in 66us (251us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188964.542328:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6464 00000100:00100000:0.0:1776188964.542569:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034c47c21 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664263744:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776188964.543499:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776188964.543515:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004490682c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664263808:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776188964.543601:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188964.543824:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664263808 00000100:00100000:0.0:1776188964.543869:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6465 00000100:00100000:0.0:1776188964.543874:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f92d37ee pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664263808:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776188964.543895:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776188964.543950:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f92d37ee pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664263808:12345-0@lo:13:osp-pre-1-0.0 Request processed in 75us (350us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188964.543956:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6465 00000100:00100000:1.0:1776188964.544038:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004490682c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664263808:0@lo:13:osp-pre-1-0.0 00010000:02020000:0.0:1776188964.700287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776188964.862674:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188964.956374:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188965.212296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188965.468303:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776188965.564478:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004490682c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664263872:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776188965.564544:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188965.564647:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664263872 00000100:00100000:1.0:1776188965.564698:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6466 00000100:00100000:1.0:1776188965.564704:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000852f310a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664263872:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776188965.564736:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776188965.564785:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000852f310a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664263872:12345-0@lo:13:osp-pre-1-1.0 Request processed in 80us (241us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188965.564801:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6466 00000100:00100000:3.0:1776188965.565291:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004490682c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664263872:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776188965.565310:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776188965.565319:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e71929fa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664263936:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776188965.565348:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188965.565418:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664263936 00000100:00100000:3.0:1776188965.565453:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6467 00000100:00100000:3.0:1776188965.565457:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003bb8357b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664263936:12345-0@lo:13:osp-pre-0-1.0 00000100:00080000:2.0:1776188965.565463:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00002000:00000024:3.0:1776188965.565468:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00080000:2.0:1776188965.565469:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776188965.565471:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776188965.565476:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9107 00000100:00080000:2.0:1776188965.565478:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776188965.565482:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776188965.565500:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008242b969 x1862459664264128/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776188965.565581:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776188965.565584:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776188965.565586:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776188965.565589:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9107 00000100:00080000:2.0:1776188965.565590:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776188965.565593:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776188965.565602:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f00e14c6 x1862459664264256/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776188965.565613:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776188965.565616:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776188965.565617:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776188965.565620:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9107 00000100:00080000:2.0:1776188965.565622:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776188965.565625:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776188965.565635:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a4832ec7 x1862459664264320/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776188965.565671:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d92ffa01 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664264064:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776188965.565700:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188965.565720:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bde4c712 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664264512:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776188965.565742:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188965.565759:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776188965.565763:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d8f74173 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664264192:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776188965.565795:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188965.565821:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664264064 00000100:00100000:2.0:1776188965.566048:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664264512 00000100:00100000:2.0:1776188965.566068:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664264192 00000100:00100000:2.0:1776188965.566085:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9625 00000100:00100000:3.0:1776188965.566133:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003bb8357b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664263936:12345-0@lo:13:osp-pre-0-1.0 Request processed in 675us (784us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188965.566140:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6467 00000100:00100000:3.0:1776188965.566161:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008242b969 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664264128:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776188965.566212:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188965.566222:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f00e14c6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664264256:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776188965.566256:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188965.566264:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a4832ec7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664264320:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776188965.566285:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188965.566310:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664264128 00000100:00100000:3.0:1776188965.566323:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664264256 00000100:00100000:3.0:1776188965.566331:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9626 00000100:00100000:3.0:1776188965.566335:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cb90bb32 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664264512:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188965.566370:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cb90bb32 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664264512:12345-0@lo:400:kworker/u8:1.0 Request processed in 34us (627us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188965.566376:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9626 00000100:00100000:3.0:1776188965.566388:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9627 00000100:00100000:3.0:1776188965.566392:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008666a42e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664264192:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776188965.566395:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006392c95c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664264064:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776188965.566418:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006392c95c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664264064:12345-0@lo:400:kworker/u8:1.0 Request processed in 329us (718us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188965.566418:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008666a42e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664264192:12345-0@lo:400:kworker/u8:1.0 Request processed in 26us (623us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188965.566424:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9625 00000100:00100000:3.0:1776188965.566437:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9627 00000100:00100000:0.0:1776188965.566488:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57122 00000100:00100000:0.0:1776188965.566493:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000829e500a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664264128:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776188965.566515:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776188965.566550:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000829e500a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664264128:12345-0@lo:38:kworker/u8:1.0 Request processed in 57us (339us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188965.566556:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57122 00000100:00100000:1.0:1776188965.566601:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b43d2cdc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664264384:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188965.566629:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188965.566646:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1776188965.566651:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b7a82fb0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664264000:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188965.566673:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188965.566684:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a0b45fb4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664264448:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188965.566699:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188965.566722:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57123 00000100:00100000:1.0:1776188965.566725:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b2d81ea0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664264256:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776188965.566743:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776188965.566771:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b2d81ea0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664264256:12345-0@lo:38:kworker/u8:1.0 Request processed in 45us (515us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188965.566779:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57123 00000100:00100000:3.0:1776188965.567999:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664264320 00000100:00100000:3.0:1776188965.568038:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9040 00000100:00100000:3.0:1776188965.568042:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000034c47c21 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664264320:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776188965.568055:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188965.568099:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000034c47c21 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664264320:12345-0@lo:38:kworker/u8:1.0 Request processed in 56us (1813us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188965.568105:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9040 00000100:00100000:3.0:1776188965.568147:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d8f74173 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664264192:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188965.568161:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d92ffa01 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664264064:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188965.568170:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bde4c712 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664264512:0@lo:400:kworker/u8:1.0 00000100:00080000:3.0:1776188965.568354:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188965.568358:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188965.568361:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008242b969 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664264128:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776188965.568371:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188965.568374:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188965.568377:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f00e14c6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664264256:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776188965.568385:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188965.568388:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188965.568391:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a4832ec7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664264320:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776188965.568518:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e71929fa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664263936:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776188965.568553:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664264384 00000100:00100000:3.0:1776188965.568586:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664264448 00000100:00100000:3.0:1776188965.568605:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8282 00000100:00100000:3.0:1776188965.568608:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000576bf6bf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664264384:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188965.568650:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000576bf6bf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664264384:12345-0@lo:400:kworker/u8:1.0 Request processed in 40us (2020us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188965.568657:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8282 00000100:00100000:3.0:1776188965.568669:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8283 00000100:00100000:3.0:1776188965.568672:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bd5316a5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664264448:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188965.568697:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bd5316a5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664264448:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (1997us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188965.568701:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8283 00000100:00100000:3.0:1776188965.568722:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664264000 00000100:00100000:0.0:1776188965.568967:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a0b45fb4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664264448:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188965.568967:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1291 00000100:00100000:3.0:1776188965.568971:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000afb92d61 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664264000:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188965.568978:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b43d2cdc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664264384:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188965.568999:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000afb92d61 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664264000:12345-0@lo:400:kworker/u8:1.0 Request processed in 28us (2327us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188965.569003:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b7a82fb0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664264000:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188965.569005:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1291 00010000:02020000:0.0:1776188965.724706:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188965.884660:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188965.980293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188966.236289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188966.492638:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188966.748315:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188966.912275:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188967.004993:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188967.260296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188967.517355:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188967.772331:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188967.935532:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188968.028246:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188968.284412:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188968.540436:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188968.796893:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776188968.959833:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188969.055503:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188969.308299:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776188969.338824:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188969.338865:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188969.338902:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188969.338999:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188969.339000:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654935872 00000100:00100000:2.0:1776188969.339030:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654936064 00000100:00100000:3.0:1776188969.339069:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1292 00000100:00100000:2.0:1776188969.339070:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8284 00000100:00100000:2.0:1776188969.339082:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001ae8c4d3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2280:x1862459654936064:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1776188969.339083:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008f85e8f7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654935872:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776188969.339209:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654936000 00000100:00100000:0.0:1776188969.339231:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654935936 00000100:00100000:0.0:1776188969.339250:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9628 00000100:00100000:0.0:1776188969.339254:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e2a0d74f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654935936:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1776188969.339319:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57124 00000100:00100000:1.0:1776188969.339325:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bf08c9bd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654936000:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:1.0:1776188969.339339:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776188969.339407:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001ae8c4d3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2280:x1862459654936064:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 329us (408us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188969.339415:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8284 00000100:00100000:3.0:1776188969.339831:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008f85e8f7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654935872:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 753us (927us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188969.339839:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1292 00000100:00100000:0.0:1776188969.340126:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e2a0d74f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654935936:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 869us (1260us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188969.340134:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9628 00000100:00100000:1.0:1776188969.340262:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bf08c9bd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654936000:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 935us (1445us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188969.340269:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57124 00010000:02020000:0.0:1776188969.564283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188969.660879:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cc1af34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664264576:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776188969.660951:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188969.661302:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664264576 00000100:00100000:1.0:1776188969.661373:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6468 00000100:00100000:1.0:1776188969.661381:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e71929fa pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664264576:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776188969.661421:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776188969.661501:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e71929fa pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664264576:12345-0@lo:13:osp-pre-0-0.0 Request processed in 120us (550us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188969.661508:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6468 00000100:00100000:0.0:1776188969.661559:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cc1af34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664264576:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776188969.662430:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f40104a2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664264640:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776188969.662494:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188969.662559:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664264640 00000100:00100000:1.0:1776188969.662604:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6469 00000100:00100000:1.0:1776188969.662611:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f8f9cfc6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664264640:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776188969.662643:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776188969.662708:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f8f9cfc6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664264640:12345-0@lo:13:osp-pre-1-0.0 Request processed in 96us (216us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188969.662715:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6469 00000100:00100000:0.0:1776188969.662819:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f40104a2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664264640:0@lo:13:osp-pre-1-0.0 00010000:02020000:0.0:1776188969.820605:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188969.980970:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188970.076330:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188970.332338:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188970.588389:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:2.0:1776188970.684648:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776188970.684658:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776188970.684662:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776188970.684672:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9112 00000100:00080000:2.0:1776188970.684679:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00080000:2.0:1776188970.684685:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776188970.684710:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008db7f874 x1862459664264832/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776188970.684734:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776188970.684737:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776188970.684739:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776188970.684742:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9112 00000100:00080000:2.0:1776188970.684745:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00080000:2.0:1776188970.684748:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776188970.684762:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b51bffd7 x1862459664264960/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776188970.684770:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776188970.684774:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776188970.684775:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776188970.684778:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9112 00000100:00080000:2.0:1776188970.684780:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 6s 00000100:00080000:2.0:1776188970.684800:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776188970.684814:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000022056a99 x1862459664265024/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776188970.684923:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dde983b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664264704:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188970.685018:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188970.685056:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ac30dfaa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664265152:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188970.685099:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188970.685124:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1776188970.685130:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000694ac12a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664265088:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188970.685152:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188970.685165:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046646cdd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664265344:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188970.685212:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188970.685310:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664265152 00000100:00100000:0.0:1776188970.685368:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664265088 00000100:00100000:0.0:1776188970.685386:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664265344 00000100:00100000:0.0:1776188970.685406:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8285 00000100:00100000:0.0:1776188970.685412:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004490682c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664265152:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188970.685429:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d53a33cc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664264768:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188970.685464:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188970.685474:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004490682c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664265152:12345-0@lo:400:kworker/u8:1.0 Request processed in 62us (376us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188970.685482:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8285 00000100:00100000:3.0:1776188970.685495:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007f9bf6ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664265216:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188970.685498:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8286 00000100:00100000:0.0:1776188970.685501:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000549bc579 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664265088:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188970.685516:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188970.685523:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000549bc579 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664265088:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (371us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188970.685529:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8286 00000100:00100000:3.0:1776188970.685532:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1776188970.685537:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe93ff8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664264896:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188970.685539:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8287 00000100:00100000:0.0:1776188970.685543:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004be216e7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664265344:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188970.685557:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188970.685562:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004be216e7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664265344:12345-0@lo:400:kworker/u8:1.0 Request processed in 19us (351us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188970.685568:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8287 00000100:00100000:3.0:1776188970.685570:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c1bd7ae7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664265280:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188970.685590:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188970.685613:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008db7f874 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664264832:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776188970.685615:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664265280 00000100:00100000:0.0:1776188970.685636:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8288 00000100:00100000:3.0:1776188970.685637:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188970.685640:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009be3a12f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664265280:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188970.685649:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b51bffd7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664264960:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776188970.685668:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188970.685669:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009be3a12f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664265280:12345-0@lo:400:kworker/u8:1.0 Request processed in 29us (80us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188970.685675:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8288 00000100:00100000:3.0:1776188970.685680:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000022056a99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664265024:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776188970.685743:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664264704 00000100:00100000:2.0:1776188970.685777:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1293 00000100:00100000:2.0:1776188970.685781:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000034f05964 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664264704:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188970.685827:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c1bd7ae7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664265280:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188970.685864:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000001d1f947 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664265408:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776188970.685941:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664264768 00000100:00100000:1.0:1776188970.685972:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664265216 00000100:00100000:1.0:1776188970.685983:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664264896 00000100:00100000:1.0:1776188970.685993:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664264832 00000100:00100000:1.0:1776188970.686003:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664264960 00000100:00100000:1.0:1776188970.686013:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9629 00000100:00100000:1.0:1776188970.686016:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001f3c9287 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664264768:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188970.693236:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188970.693651:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9630 00000100:00100000:3.0:1776188970.693664:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000739703f0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664265216:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188970.693736:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000739703f0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664265216:12345-0@lo:400:kworker/u8:1.0 Request processed in 75us (8219us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188970.693745:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9630 00000100:00100000:3.0:1776188970.693777:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9631 00000100:00100000:3.0:1776188970.693783:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000327b9bbd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664264896:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188970.693815:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000327b9bbd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664264896:12345-0@lo:400:kworker/u8:1.0 Request processed in 32us (8258us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188970.693821:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9631 00000100:00100000:3.0:1776188970.693835:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57125 00000100:00100000:3.0:1776188970.693840:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003d3c789d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664264832:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776188970.693860:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188970.693897:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003d3c789d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664264832:12345-0@lo:38:kworker/u8:1.0 Request processed in 56us (8259us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188970.693903:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57125 00000100:00100000:3.0:1776188970.693988:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57126 00000100:00100000:3.0:1776188970.694008:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000ab1bdf7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664264960:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776188970.694029:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188970.694120:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000ab1bdf7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664264960:12345-0@lo:38:kworker/u8:1.0 Request processed in 115us (8450us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188970.694127:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57126 00000100:00100000:3.0:1776188970.694244:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664265024 00000100:00100000:3.0:1776188970.694279:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9041 00000100:00100000:3.0:1776188970.694283:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a7b8e04c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664265024:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776188970.694292:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188970.694319:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a7b8e04c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664265024:12345-0@lo:38:kworker/u8:1.0 Request processed in 35us (1096us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188970.694325:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9041 00000100:00080000:3.0:1776188970.694433:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188970.694438:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188970.694442:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008db7f874 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664264832:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776188970.694453:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188970.694456:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188970.694458:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b51bffd7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664264960:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776188970.694465:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188970.694468:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188970.694471:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000022056a99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664265024:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776188970.694510:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000055c776dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664265472:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776188970.694544:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188970.694580:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe93ff8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664264896:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188970.694591:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007f9bf6ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664265216:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188970.694981:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664265472 00000100:00100000:3.0:1776188970.695015:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6470 00000100:00100000:3.0:1776188970.695019:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008b77e02f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664265472:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776188970.695036:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776188970.695070:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008b77e02f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664265472:12345-0@lo:13:osp-pre-1-1.0 Request processed in 50us (527us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188970.695090:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6470 00000100:00100000:3.0:1776188970.695210:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000055c776dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664265472:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776188970.699004:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000034f05964 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664264704:12345-0@lo:400:kworker/u8:1.0 Request processed in 13209us (13974us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188970.699027:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1293 00000100:00100000:0.0:1776188970.703622:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188970.703692:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664265408 00000100:00100000:0.0:1776188970.703737:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000694ac12a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664265088:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188970.703759:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046646cdd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664265344:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188970.703771:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dde983b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664264704:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188970.703783:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ac30dfaa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664265152:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188970.704219:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6471 00000100:00100000:3.0:1776188970.704237:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000045a10d8d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664265408:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776188970.704266:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776188970.704349:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000045a10d8d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664265408:12345-0@lo:13:osp-pre-0-1.0 Request processed in 112us (738us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188970.704356:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6471 00000100:00100000:2.0:1776188970.704461:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000001d1f947 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664265408:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776188970.711153:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d53a33cc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664264768:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188970.711714:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001f3c9287 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664264768:12345-0@lo:400:kworker/u8:1.0 Request processed in 24750us (25303us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188970.711731:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9629 00010000:02020000:0.0:1776188970.844326:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188971.006126:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188971.100796:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188971.356283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188971.612375:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188971.868341:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188972.029536:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188972.124483:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188972.380502:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188972.636313:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188972.892586:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776188973.052626:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188973.148312:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188973.404302:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188973.660335:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188973.917982:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188974.076739:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188974.172588:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188974.428335:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776188974.461869:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188974.461922:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188974.461965:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188974.462024:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188974.462076:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654936320 00000100:00100000:2.0:1776188974.462141:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8289 00000100:00100000:2.0:1776188974.462150:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003258ba66 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654936320:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1776188974.462303:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003258ba66 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654936320:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 153us (279us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188974.462311:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8289 00000100:00100000:3.0:1776188974.462765:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654936256 00000100:00100000:3.0:1776188974.462808:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654936192 00000100:00100000:3.0:1776188974.462836:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9632 00000100:00100000:3.0:1776188974.462841:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001686e875 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654936192:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1776188974.462890:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001686e875 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654936192:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 42us (963us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188974.462897:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9632 00000100:00100000:3.0:1776188974.462912:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57127 00000100:00100000:3.0:1776188974.462916:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d53a33cc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654936256:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:3.0:1776188974.462933:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776188974.462962:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d53a33cc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654936256:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 45us (1105us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188974.462968:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57127 00000100:00100000:0.0:1776188974.463130:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654936128 00000100:00100000:0.0:1776188974.463168:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1294 00000100:00100000:0.0:1776188974.463197:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d6b85e7e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654936128:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776188974.463227:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d6b85e7e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654936128:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 54us (1263us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188974.463234:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1294 00010000:02020000:0.0:1776188974.684503:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776188974.789625:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000624e744d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664265536:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776188974.789715:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188974.790970:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776188974.791003:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000055c776dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664265600:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776188974.791062:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188974.799469:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664265536 00000100:00100000:2.0:1776188974.818681:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664265600 00000100:00100000:2.0:1776188974.819053:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6472 00000100:00100000:2.0:1776188974.819061:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0fdb037 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664265536:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776188974.819107:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776188974.819746:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0fdb037 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664265536:12345-0@lo:13:osp-pre-0-0.0 Request processed in 681us (30029us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188974.819757:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6472 00000100:00100000:2.0:1776188974.819783:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6473 00000100:00100000:2.0:1776188974.819788:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004abaab4f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664265600:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776188974.819812:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776188974.819853:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004abaab4f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664265600:12345-0@lo:13:osp-pre-1-0.0 Request processed in 65us (28792us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188974.819860:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6473 00000100:00100000:2.0:1776188974.819938:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000624e744d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664265536:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776188974.820025:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000055c776dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664265600:0@lo:13:osp-pre-1-0.0 00010000:02020000:0.0:1776188974.940330:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188975.102530:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188975.196393:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188975.452319:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188975.708753:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776188975.804686:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776188975.804716:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000030d1626f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664265664:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776188975.804834:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188975.805021:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fda571 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664265728:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776188975.805056:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776188975.805070:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1776188975.805073:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00080000:1.0:1776188975.805076:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1776188975.805078:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602ee85e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664265792:0@lo:400:kworker/u8:0.0 00000100:00080000:1.0:1776188975.805080:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776188975.805086:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9117 00000100:00080000:1.0:1776188975.805091:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776188975.805097:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188975.805108:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188975.805120:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006145bcbc x1862459664265920/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776188975.805145:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1776188975.805145:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664265792 00000100:00080000:1.0:1776188975.805149:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776188975.805150:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776188975.805154:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9117 00000100:00080000:1.0:1776188975.805155:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776188975.805159:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776188975.805168:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001929a2c4 x1862459664266048/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776188975.805193:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776188975.805196:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776188975.805197:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776188975.805200:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9117 00000100:00080000:1.0:1776188975.805202:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776188975.805205:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776188975.805214:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1295 00000100:00100000:1.0:1776188975.805217:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000250cea25 x1862459664266112/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776188975.805220:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004c446be7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664265792:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188975.805257:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000058c450e1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664266176:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188975.805267:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004c446be7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664265792:12345-0@lo:400:kworker/u8:0.0 Request processed in 46us (158us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188975.805274:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1295 00000100:00100000:1.0:1776188975.805286:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188975.805323:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602ee85e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664265792:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188975.805347:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776188975.805352:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001fc79665 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664266240:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188975.805375:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188975.805401:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664266176 00000100:00100000:1.0:1776188975.805431:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664266240 00000100:00100000:0.0:1776188975.805508:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664265664 00000100:00100000:0.0:1776188975.805537:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664265728 00000100:00100000:1.0:1776188975.805548:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8290 00000100:00100000:0.0:1776188975.805552:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6474 00000100:00100000:1.0:1776188975.805553:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002bf77dc0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664266176:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188975.805556:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c2d64a45 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664265664:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776188975.805574:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776188975.805593:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002bf77dc0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664266176:12345-0@lo:400:kworker/u8:0.0 Request processed in 40us (306us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188975.805600:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8290 00000100:00100000:0.0:1776188975.805608:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c2d64a45 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664265664:12345-0@lo:13:osp-pre-1-1.0 Request processed in 52us (776us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188975.805614:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8291 00000100:00100000:1.0:1776188975.805617:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000029604218 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664266240:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188975.805641:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000029604218 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664266240:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (266us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188975.805647:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8291 00000100:00100000:1.0:1776188975.805678:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001fc79665 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664266240:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188975.805688:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000058c450e1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664266176:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188975.805706:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6474 00000100:00100000:1.0:1776188975.805718:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006145bcbc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664265920:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776188975.805719:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6475 00000100:00100000:0.0:1776188975.805722:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000817f7178 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664265728:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776188975.805734:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776188975.805747:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188975.805761:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001929a2c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664266048:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776188975.805762:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000817f7178 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664265728:12345-0@lo:13:osp-pre-0-1.0 Request processed in 39us (706us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188975.805768:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6475 00000100:00100000:0.0:1776188975.805794:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fda571 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664265728:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776188975.805824:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000075d6a9fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664265856:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188975.806078:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188975.806156:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664265920 00000100:00100000:2.0:1776188975.806203:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664266048 00000100:00100000:2.0:1776188975.806222:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57128 00000100:00100000:2.0:1776188975.806226:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000549bb791 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664265920:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776188975.806242:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776188975.806338:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57129 00000100:00100000:0.0:1776188975.806342:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000e1dd712 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664266048:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776188975.806349:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188975.815378:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000154c017b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664265984:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188975.822659:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000250cea25 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664266112:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776188975.822758:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188975.822834:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664266112 00000100:00100000:1.0:1776188975.822880:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9042 00000100:00100000:1.0:1776188975.822886:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000080eb3f20 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664266112:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776188975.822910:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776188975.822978:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000080eb3f20 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664266112:12345-0@lo:38:kworker/u8:0.0 Request processed in 91us (222us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188975.822991:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9042 00000100:00080000:1.0:1776188975.823041:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776188975.823047:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776188975.823052:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000250cea25 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664266112:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776188975.823128:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188975.823157:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bbf4e716 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664266304:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188975.823216:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188975.823258:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000030d1626f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664265664:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776188975.823290:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664265856 00000100:00100000:3.0:1776188975.823720:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188975.823765:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664266304 00000100:00100000:3.0:1776188975.823812:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664265984 00000100:00100000:3.0:1776188975.823828:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9633 00000100:00100000:3.0:1776188975.823833:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005dd3a2a0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664266304:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188975.823896:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005dd3a2a0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664266304:12345-0@lo:400:kworker/u8:0.0 Request processed in 62us (680us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188975.823903:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9633 00000100:00100000:1.0:1776188975.823913:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9634 00000100:00100000:3.0:1776188975.823916:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9635 00000100:00100000:1.0:1776188975.823917:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008b4c364e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664265984:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188975.823919:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e6bde712 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664265856:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188975.823944:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e6bde712 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664265856:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (817us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188975.823948:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008b4c364e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664265984:12345-0@lo:400:kworker/u8:0.0 Request processed in 31us (230us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188975.823954:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9635 00000100:00100000:3.0:1776188975.823998:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000154c017b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664265984:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188975.824041:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9634 00000100:00100000:1.0:1776188975.824207:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000075d6a9fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664265856:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188975.824223:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bbf4e716 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664266304:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188975.824865:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000549bb791 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664265920:12345-0@lo:38:kworker/u8:0.0 Request processed in 18635us (19115us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776188975.824875:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57128 00000100:00080000:1.0:1776188975.825085:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776188975.825090:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776188975.825094:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006145bcbc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664265920:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776188975.830657:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000e1dd712 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664266048:12345-0@lo:38:kworker/u8:0.0 Request processed in 24300us (24862us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188975.830679:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57129 00000100:00080000:1.0:1776188975.831141:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776188975.831148:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776188975.831156:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001929a2c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664266048:0@lo:38:kworker/u8:0.0 00010000:02020000:0.0:1776188975.964373:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188976.124534:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188976.220355:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188976.476351:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188976.732892:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188976.988554:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776188977.149914:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188977.244419:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188977.500478:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188977.756291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188978.012429:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188978.172564:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188978.269233:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188978.524568:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188978.780409:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188979.038357:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188979.196771:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188979.292582:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188979.548937:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776188979.582505:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188979.582596:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188979.582641:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188979.582938:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654936384 00000100:00100000:3.0:1776188979.583042:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1296 00000100:00100000:3.0:1776188979.583049:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fe4b4330 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654936384:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1776188979.583122:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fe4b4330 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654936384:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 72us (624us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188979.583129:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1296 00000100:00100000:2.0:1776188979.583601:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776188979.583689:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654936448 00000100:00100000:1.0:1776188979.583728:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654936512 00000100:00100000:1.0:1776188979.583748:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9636 00000100:00100000:1.0:1776188979.583753:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000999df332 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654936448:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1776188979.583794:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000999df332 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654936448:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 39us (1153us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188979.583801:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9636 00000100:00100000:1.0:1776188979.583815:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57130 00000100:00100000:1.0:1776188979.583820:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000041c7e233 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654936512:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:1.0:1776188979.583834:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776188979.583860:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000041c7e233 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654936512:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 40us (261us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188979.583866:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57130 00000100:00100000:0.0:1776188979.584634:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654936576 00000100:00100000:0.0:1776188979.584702:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8292 00000100:00100000:0.0:1776188979.584706:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000091ef61c6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654936576:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776188979.584756:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000091ef61c6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654936576:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 49us (2161us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188979.584763:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8292 00010000:02020000:0.0:1776188979.804297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776188979.900964:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776188979.900986:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc7962a4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664266432:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776188979.901048:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188979.901095:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664266432 00000100:00100000:3.0:1776188979.901134:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6476 00000100:00100000:3.0:1776188979.901139:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000381790a0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664266432:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776188979.901217:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776188979.901253:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000381790a0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664266432:12345-0@lo:13:osp-pre-1-0.0 Request processed in 113us (206us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188979.901260:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6476 00000100:00100000:3.0:1776188979.901302:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc7962a4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664266432:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776188979.901631:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776188979.901653:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fda571 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664266368:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776188979.901699:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188979.901943:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664266368 00000100:00100000:3.0:1776188979.901984:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6477 00000100:00100000:3.0:1776188979.901989:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d781d9c8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664266368:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776188979.902006:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776188979.902055:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d781d9c8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664266368:12345-0@lo:13:osp-pre-0-0.0 Request processed in 65us (355us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188979.902062:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6477 00000100:00100000:2.0:1776188979.902211:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fda571 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664266368:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776188980.060511:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188980.221704:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188980.316278:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188980.572280:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188980.828419:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:1.0:1776188980.924704:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776188980.924711:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776188980.924714:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776188980.924721:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9122 00000100:00080000:1.0:1776188980.924725:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:1.0:1776188980.924730:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776188980.924747:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bde17028 x1862459664266624/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776188980.924765:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776188980.924768:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776188980.924769:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776188980.924772:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9122 00000100:00080000:1.0:1776188980.924774:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00080000:1.0:1776188980.924777:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776188980.924784:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000063072b2b x1862459664266752/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776188980.924790:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776188980.924792:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776188980.924793:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776188980.924796:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9122 00000100:00080000:1.0:1776188980.924797:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00080000:1.0:1776188980.924799:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776188980.924806:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000994add5b x1862459664266816/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776188980.924876:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e718565 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664266688:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188980.924933:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188980.924955:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c6e8ce8b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664267072:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188980.924975:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188980.924992:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1776188980.924996:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a87ab8e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664266560:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188980.925012:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188980.925020:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000174c978a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664267008:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188980.925034:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188980.925053:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bde17028 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664266624:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776188980.925070:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188980.925080:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000063072b2b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664266752:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776188980.925095:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188980.925102:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000994add5b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664266816:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776188980.925119:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188980.925145:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664266688 00000100:00100000:1.0:1776188980.925196:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664266560 00000100:00100000:1.0:1776188980.925294:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664267008 00000100:00100000:1.0:1776188980.925303:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664266624 00000100:00100000:1.0:1776188980.925310:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664266752 00000100:00100000:1.0:1776188980.925319:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9637 00000100:00100000:1.0:1776188980.925323:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000062fd23f7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664266688:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188980.925364:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000062fd23f7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664266688:12345-0@lo:400:kworker/u8:1.0 Request processed in 40us (432us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188980.925370:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9637 00000100:00100000:1.0:1776188980.925384:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9638 00000100:00100000:1.0:1776188980.925386:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000005505a39 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664266560:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188980.925404:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000005505a39 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664266560:12345-0@lo:400:kworker/u8:1.0 Request processed in 17us (392us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188980.925409:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9638 00000100:00100000:1.0:1776188980.925418:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9639 00000100:00100000:1.0:1776188980.925421:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000061aa2eb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664267008:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188980.925440:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000061aa2eb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664267008:12345-0@lo:400:kworker/u8:1.0 Request processed in 18us (406us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188980.925445:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9639 00000100:00100000:1.0:1776188980.925456:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57131 00000100:00100000:1.0:1776188980.925459:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b5628367 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664266624:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776188980.925472:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776188980.925501:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b5628367 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664266624:12345-0@lo:38:kworker/u8:1.0 Request processed in 41us (430us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188980.925506:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57131 00000100:00100000:1.0:1776188980.925516:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57132 00000100:00100000:1.0:1776188980.925519:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000071b1847 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664266752:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776188980.925525:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776188980.925545:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000071b1847 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664266752:12345-0@lo:38:kworker/u8:1.0 Request processed in 25us (450us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188980.925550:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57132 00000100:00100000:1.0:1776188980.925571:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664266816 00000100:00100000:1.0:1776188980.925587:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9043 00000100:00100000:1.0:1776188980.925590:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001929a2c4 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664266816:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776188980.925597:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776188980.925620:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001929a2c4 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664266816:12345-0@lo:38:kworker/u8:1.0 Request processed in 29us (501us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188980.925626:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9043 00000100:00080000:1.0:1776188980.925656:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776188980.925660:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776188980.925663:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bde17028 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664266624:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776188980.925671:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776188980.925673:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776188980.925675:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000063072b2b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664266752:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776188980.925695:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776188980.925697:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776188980.925699:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000994add5b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664266816:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776188980.925735:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a87ab8e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664266560:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188980.925742:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000174c978a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664267008:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188980.925749:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e718565 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664266688:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776188980.925834:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004d465759 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664266880:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776188980.925873:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188980.925893:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e6bd251 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664267136:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776188980.925911:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188980.925924:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1776188980.925928:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fd2b3fb0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664266496:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776188980.925949:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188980.925959:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fb7c0d6f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664266944:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776188980.925976:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188980.926001:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664267072 00000100:00100000:2.0:1776188980.926026:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664266880 00000100:00100000:2.0:1776188980.926040:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664267136 00000100:00100000:2.0:1776188980.926051:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664266944 00000100:00100000:2.0:1776188980.926064:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8293 00000100:00100000:2.0:1776188980.926067:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003eb43bde pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664267072:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776188980.926099:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003eb43bde pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664267072:12345-0@lo:400:kworker/u8:1.0 Request processed in 31us (1123us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188980.926105:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8293 00000100:00100000:2.0:1776188980.926117:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8294 00000100:00100000:2.0:1776188980.926120:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000079a89d96 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664266880:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776188980.926148:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000079a89d96 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664266880:12345-0@lo:400:kworker/u8:1.0 Request processed in 27us (275us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188980.926153:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8294 00000100:00100000:2.0:1776188980.926163:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8295 00000100:00100000:2.0:1776188980.926165:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000058e0433d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664267136:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776188980.926228:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000058e0433d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664267136:12345-0@lo:400:kworker/u8:1.0 Request processed in 61us (316us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188980.926234:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8295 00000100:00100000:2.0:1776188980.926244:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8296 00000100:00100000:2.0:1776188980.926247:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006392c95c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664266944:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776188980.926269:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006392c95c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664266944:12345-0@lo:400:kworker/u8:1.0 Request processed in 21us (293us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188980.926273:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8296 00000100:00100000:2.0:1776188980.926307:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c6e8ce8b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664267072:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776188980.926328:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fb7c0d6f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664266944:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776188980.926336:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004d465759 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664266880:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776188980.926344:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e6bd251 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664267136:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188980.926561:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664266496 00000100:00100000:3.0:1776188980.926585:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1297 00000100:00100000:3.0:1776188980.926589:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000459e616f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664266496:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188980.926615:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000459e616f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664266496:12345-0@lo:400:kworker/u8:1.0 Request processed in 26us (666us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188980.926620:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1297 00000100:00100000:3.0:1776188980.926654:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fd2b3fb0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664266496:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188980.926664:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776188980.926668:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc7962a4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664267200:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776188980.926689:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188980.926709:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664267200 00000100:00100000:3.0:1776188980.926729:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6478 00000100:00100000:3.0:1776188980.926732:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000064cbacbf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664267200:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776188980.926744:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776188980.926767:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000064cbacbf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664267200:12345-0@lo:13:osp-pre-1-1.0 Request processed in 34us (78us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188980.926772:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6478 00000100:00100000:3.0:1776188980.926794:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc7962a4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664267200:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776188980.927170:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc7962a4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664267264:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776188980.927228:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188980.927376:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664267264 00000100:00100000:3.0:1776188980.927408:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6479 00000100:00100000:3.0:1776188980.927412:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000b229330 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664267264:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776188980.927423:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776188980.927456:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000b229330 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664267264:12345-0@lo:13:osp-pre-0-1.0 Request processed in 43us (228us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188980.927461:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6479 00000100:00100000:0.0:1776188980.927534:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc7962a4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664267264:0@lo:13:osp-pre-0-1.0 00010000:02020000:0.0:1776188981.084309:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776188981.244489:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188981.340307:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188981.596309:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188981.852346:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188982.108403:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188982.268681:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188982.364284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188982.620279:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188982.876275:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188983.132315:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188983.292870:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188983.391295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188983.645970:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188983.900338:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188984.156530:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188984.316618:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188984.419136:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188984.676319:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776188984.698936:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188984.699378:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654936640 00000100:00100000:3.0:1776188984.699436:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1298 00000100:00100000:3.0:1776188984.699442:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000de62cc6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654936640:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1776188984.699510:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000de62cc6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654936640:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 68us (582us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188984.699518:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1298 00000100:00100000:1.0:1776188984.715614:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776188984.715657:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776188984.715865:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188984.716528:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654936832 00000100:00100000:2.0:1776188984.716603:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8297 00000100:00100000:2.0:1776188984.716609:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000075d6a9fc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654936832:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1776188984.716713:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000075d6a9fc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654936832:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 102us (848us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188984.716720:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8297 00000100:00100000:0.0:1776188984.720792:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654936704 00000100:00100000:0.0:1776188984.720876:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654936768 00000100:00100000:0.0:1776188984.720912:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9640 00000100:00100000:0.0:1776188984.720919:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006145bcbc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654936704:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776188984.721025:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006145bcbc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654936704:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 106us (5419us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188984.721033:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9640 00000100:00100000:0.0:1776188984.721056:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57133 00000100:00100000:0.0:1776188984.721060:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bbf4e716 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654936768:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:0.0:1776188984.721075:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776188984.721102:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bbf4e716 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654936768:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 41us (5445us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188984.721108:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57133 00010000:02020000:0.0:1776188984.935368:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776188985.020811:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000154c017b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664267328:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776188985.020883:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188985.021922:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664267328 00000100:00100000:1.0:1776188985.022003:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6480 00000100:00100000:1.0:1776188985.022009:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003614bde1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664267328:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776188985.022048:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776188985.022104:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003614bde1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664267328:12345-0@lo:13:osp-pre-1-0.0 Request processed in 94us (1221us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188985.022111:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6480 00000100:00100000:1.0:1776188985.022161:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000154c017b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664267328:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776188985.023070:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bec72d87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664267392:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776188985.023121:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188985.023427:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664267392 00000100:00100000:0.0:1776188985.023473:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6481 00000100:00100000:0.0:1776188985.023478:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004f7a6a77 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664267392:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776188985.023502:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776188985.023543:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004f7a6a77 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664267392:12345-0@lo:13:osp-pre-0-0.0 Request processed in 64us (422us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188985.023550:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6481 00000100:00100000:0.0:1776188985.023586:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bec72d87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664267392:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776188985.188993:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776188985.342524:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188985.444598:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188985.700384:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188985.956304:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776188986.044415:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e50a27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664267456:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776188986.044497:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188986.044632:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bec72d87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664267520:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776188986.044663:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188986.049916:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664267456 00000100:00100000:3.0:1776188986.049942:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664267520 00000100:00100000:3.0:1776188986.049993:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6482 00000100:00100000:2.0:1776188986.050001:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6483 00000100:00100000:3.0:1776188986.050002:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ac30dfaa pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664267456:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776188986.050010:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000041c094ed pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664267520:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776188986.050035:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00002000:00000024:3.0:1776188986.050044:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776188986.050115:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000041c094ed pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664267520:12345-0@lo:13:osp-pre-0-1.0 Request processed in 107us (5452us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188986.050118:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ac30dfaa pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664267456:12345-0@lo:13:osp-pre-1-1.0 Request processed in 118us (5621us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188986.050128:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6483 00000100:00100000:3.0:1776188986.050148:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6482 00000100:00080000:1.0:1776188986.050163:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776188986.050169:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776188986.050193:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776188986.050200:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9127 00000100:00080000:1.0:1776188986.050203:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1776188986.050210:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e50a27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664267456:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776188986.050229:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776188986.050237:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000250cea25 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664267648:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188986.050268:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188986.050296:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664267648 00000100:00100000:2.0:1776188986.050321:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9641 00000100:00100000:2.0:1776188986.050338:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dde983b4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664267648:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188986.050372:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dde983b4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664267648:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (104us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188986.050380:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9641 00000100:00100000:3.0:1776188986.050405:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000250cea25 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664267648:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188986.050449:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000154c017b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664267584:0@lo:400:kworker/u8:0.0 00000100:00080000:1.0:1776188986.050455:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776188986.050481:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188986.050487:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000058c450e1 x1862459664267712/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776188986.050524:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776188986.050528:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776188986.050529:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776188986.050533:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9127 00000100:00080000:1.0:1776188986.050535:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776188986.050538:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776188986.050549:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000602ee85e x1862459664267840/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776188986.050558:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776188986.050561:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776188986.050563:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776188986.050566:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9127 00000100:00080000:1.0:1776188986.050569:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776188986.050572:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776188986.050584:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000250cea25 x1862459664267904/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776188986.050590:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bec72d87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664267520:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776188986.050610:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fd2b3fb0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664267968:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188986.050638:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188986.050670:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776188986.050672:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664267968 00000100:00100000:3.0:1776188986.050682:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001fc79665 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664267776:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188986.050705:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8298 00000100:00100000:0.0:1776188986.050712:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bec72d87 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664267968:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188986.050718:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188986.050730:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000058c450e1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664267712:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776188986.050769:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bec72d87 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664267968:12345-0@lo:400:kworker/u8:0.0 Request processed in 56us (129us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188986.050776:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664267776 00000100:00100000:2.0:1776188986.050777:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188986.050786:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8298 00000100:00100000:2.0:1776188986.050802:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602ee85e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664267840:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188986.050806:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9642 00000100:00100000:3.0:1776188986.050812:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000094e50a27 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664267776:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188986.050819:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664267712 00000100:00100000:2.0:1776188986.050832:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188986.050858:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000250cea25 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664267904:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776188986.050881:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188986.051006:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000094e50a27 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664267776:12345-0@lo:400:kworker/u8:0.0 Request processed in 193us (290us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188986.051013:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9642 00000100:00100000:0.0:1776188986.051463:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664267840 00000100:00100000:0.0:1776188986.051484:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57134 00000100:00100000:0.0:1776188986.051488:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000046646cdd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664267712:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776188986.051503:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776188986.051543:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000046646cdd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664267712:12345-0@lo:38:kworker/u8:0.0 Request processed in 54us (770us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188986.051550:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57134 00000100:00100000:2.0:1776188986.051557:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57135 00000100:00100000:2.0:1776188986.051563:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000694ac12a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664267840:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776188986.051573:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188986.051581:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664267584 00000100:00080000:0.0:1776188986.051584:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776188986.051589:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776188986.051594:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000058c450e1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664267712:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776188986.051609:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000694ac12a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664267840:12345-0@lo:38:kworker/u8:0.0 Request processed in 44us (777us total) trans 0 rc -19/-19 00000100:00080000:0.0:1776188986.051614:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:3.0:1776188986.051615:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1299 00000100:00080000:0.0:1776188986.051620:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776188986.051620:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57135 00000100:00100000:3.0:1776188986.051624:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000082b19b4e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664267584:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188986.051627:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602ee85e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664267840:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776188986.051667:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fd2b3fb0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664267968:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188986.051725:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664267904 00000100:00100000:0.0:1776188986.051758:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9044 00000100:00100000:0.0:1776188986.051762:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007f9bf6ea pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664267904:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776188986.051770:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776188986.051804:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007f9bf6ea pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664267904:12345-0@lo:38:kworker/u8:0.0 Request processed in 40us (921us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776188986.051804:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001fc79665 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664267776:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188986.051818:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9044 00000100:00080000:0.0:1776188986.051845:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776188986.051848:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776188986.051851:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000250cea25 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664267904:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776188986.052483:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000082b19b4e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664267584:12345-0@lo:400:kworker/u8:0.0 Request processed in 857us (2000us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188986.052493:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1299 00000100:00100000:2.0:1776188986.052501:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000154c017b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664267584:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188986.052527:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e6bd251 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664268032:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188986.052538:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776188986.052551:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004d465759 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664268096:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188986.052562:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188986.052589:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188986.052614:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664268032 00000100:00100000:1.0:1776188986.052631:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664268096 00000100:00100000:3.0:1776188986.052648:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8299 00000100:00100000:3.0:1776188986.052653:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fe93ff8e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664268032:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188986.052669:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9643 00000100:00100000:1.0:1776188986.052676:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fb7c0d6f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664268096:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188986.052696:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fe93ff8e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664268032:12345-0@lo:400:kworker/u8:0.0 Request processed in 41us (134us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188986.052704:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8299 00000100:00100000:2.0:1776188986.052716:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e6bd251 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664268032:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188986.054576:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fb7c0d6f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664268096:12345-0@lo:400:kworker/u8:0.0 Request processed in 1894us (1982us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188986.054590:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9643 00000100:00100000:2.0:1776188986.055622:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004d465759 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664268096:0@lo:400:kworker/u8:0.0 00010000:02020000:0.0:1776188986.212285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188986.364414:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188986.468388:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188986.724286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188986.980328:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188987.236373:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188987.388468:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188987.493470:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188987.748289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188988.004284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188988.260293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188988.418378:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188988.516252:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188988.772304:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188989.028503:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188989.284290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776188989.436779:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188989.541140:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188989.796318:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776188989.820378:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776188989.820475:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188989.820697:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654936960 00000100:00100000:2.0:1776188989.820775:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9644 00000100:00100000:2.0:1776188989.820781:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c6e8ce8b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654936960:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776188989.820869:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c6e8ce8b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654936960:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 73us (381us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188989.820877:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9644 00000100:00100000:3.0:1776188989.823080:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654936896 00000100:00100000:3.0:1776188989.823143:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1300 00000100:00100000:3.0:1776188989.823149:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000055012753 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654936896:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776188989.823444:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000055012753 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654936896:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 294us (3074us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188989.823453:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1300 00000100:00100000:0.0:1776188989.823760:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188989.824004:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188989.824368:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654937024 00000100:00100000:2.0:1776188989.824406:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57136 00000100:00100000:2.0:1776188989.824411:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000074e02040 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654937024:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776188989.824427:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776188989.824472:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000074e02040 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654937024:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 60us (714us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776188989.824479:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57136 00000100:00100000:3.0:1776188989.825505:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654937088 00000100:00100000:3.0:1776188989.832146:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8300 00000100:00100000:3.0:1776188989.832158:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f9e630d1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2279:x1862459654937088:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776188989.832274:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f9e630d1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2279:x1862459654937088:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 117us (8271us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188989.832282:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8300 00010000:02020000:0.0:1776188990.054285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776188990.140494:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e1f2ed1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664268160:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776188990.140561:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188990.140681:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664268160 00000100:00100000:0.0:1776188990.140730:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6484 00000100:00100000:0.0:1776188990.140736:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022056a99 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664268160:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776188990.140766:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776188990.140816:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022056a99 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664268160:12345-0@lo:13:osp-pre-0-0.0 Request processed in 79us (255us total) trans 0 rc 0/0 00000100:00100000:0.0:1776188990.140823:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6484 00000100:00100000:2.0:1776188990.141020:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e1f2ed1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664268160:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776188990.141270:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000063864031 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664268224:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776188990.141299:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188990.141460:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664268224 00000100:00100000:1.0:1776188990.141491:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6485 00000100:00100000:1.0:1776188990.141496:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e4c3b995 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664268224:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776188990.141513:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776188990.141554:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e4c3b995 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664268224:12345-0@lo:13:osp-pre-1-0.0 Request processed in 57us (254us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188990.141559:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6485 00000100:00100000:2.0:1776188990.141878:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000063864031 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664268224:0@lo:13:osp-pre-1-0.0 00010000:02020000:0.0:1776188990.308282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188990.467331:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188990.565880:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188990.820933:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188991.076241:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:2.0:1776188991.164645:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776188991.164653:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776188991.164656:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776188991.164664:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9133 00000100:00080000:2.0:1776188991.164677:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776188991.164684:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776188991.164704:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c1bd7ae7 x1862459664268480/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776188991.164726:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776188991.164730:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776188991.164731:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776188991.164734:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9133 00000100:00080000:2.0:1776188991.164736:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776188991.164739:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776188991.164750:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004abaab4f x1862459664268608/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776188991.164758:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776188991.164761:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776188991.164762:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776188991.164765:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9133 00000100:00080000:2.0:1776188991.164768:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776188991.164770:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776188991.164781:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d781d9c8 x1862459664268672/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776188991.164829:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bde4c712 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664268544:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188991.164892:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188991.164921:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c06060ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664268928:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188991.164946:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188991.164966:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:2.0:1776188991.164970:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000063864031 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664268288:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776188991.164991:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188991.165002:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ac30dfaa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664268736:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188991.165021:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188991.165035:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000023121a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664268992:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188991.165053:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188991.165088:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008db7f874 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664268416:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188991.165359:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664268288 00000100:00100000:1.0:1776188991.165403:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6486 00000100:00100000:1.0:1776188991.165409:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f4f823f8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664268288:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776188991.165428:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776188991.165473:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f4f823f8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664268288:12345-0@lo:13:osp-pre-0-1.0 Request processed in 63us (481us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188991.165480:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6486 00000100:00100000:1.0:1776188991.165523:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000063864031 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664268288:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776188991.165565:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664268544 00000100:00100000:1.0:1776188991.165590:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9645 00000100:00100000:1.0:1776188991.165594:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d4da5ec0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664268544:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188991.165630:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d4da5ec0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664268544:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (739us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188991.165649:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9645 00000100:00100000:1.0:1776188991.165685:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bde4c712 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664268544:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188991.165714:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664268928 00000100:00100000:1.0:1776188991.165737:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664268736 00000100:00100000:1.0:1776188991.165752:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664268992 00000100:00100000:1.0:1776188991.165769:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8301 00000100:00100000:1.0:1776188991.165773:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000aa8befb6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664268928:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188991.165805:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000aa8befb6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664268928:12345-0@lo:400:kworker/u8:0.0 Request processed in 31us (859us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188991.165811:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8301 00000100:00100000:1.0:1776188991.165821:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8302 00000100:00100000:1.0:1776188991.165824:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000511337f8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664268736:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188991.165847:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000511337f8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664268736:12345-0@lo:400:kworker/u8:0.0 Request processed in 22us (826us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188991.165852:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8302 00000100:00100000:1.0:1776188991.165861:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8303 00000100:00100000:1.0:1776188991.165864:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000960dca04 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664268992:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188991.165885:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000960dca04 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664268992:12345-0@lo:400:kworker/u8:0.0 Request processed in 20us (832us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188991.165891:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8303 00000100:00100000:1.0:1776188991.165913:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ac30dfaa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664268736:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188991.165923:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000023121a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664268992:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776188991.165930:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c06060ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664268928:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188991.166423:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188991.166454:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f326e309 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664268864:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188991.166482:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188991.166502:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:2.0:1776188991.166508:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b51bffd7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664268352:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188991.166534:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188991.166547:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000022056a99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664268800:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776188991.166565:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188991.166578:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008655ce15 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664269056:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776188991.166597:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188991.166626:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664268352 00000100:00100000:3.0:1776188991.166798:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664268800 00000100:00100000:3.0:1776188991.166825:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8304 00000100:00100000:3.0:1776188991.166829:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000f2a9137 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664268800:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188991.166866:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000f2a9137 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664268800:12345-0@lo:400:kworker/u8:0.0 Request processed in 36us (301us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188991.166873:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8304 00000100:00100000:3.0:1776188991.166924:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664268416 00000100:00100000:3.0:1776188991.166945:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664268864 00000100:00100000:3.0:1776188991.166959:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9646 00000100:00100000:3.0:1776188991.166962:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0b45fb4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664268416:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188991.166991:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0b45fb4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664268416:12345-0@lo:400:kworker/u8:0.0 Request processed in 27us (569us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188991.166996:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9646 00000100:00100000:3.0:1776188991.167005:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9647 00000100:00100000:3.0:1776188991.167009:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a14e98d2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664268864:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188991.167031:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a14e98d2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664268864:12345-0@lo:400:kworker/u8:0.0 Request processed in 22us (549us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188991.167036:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9647 00000100:00100000:3.0:1776188991.167051:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664269056 00000100:00100000:3.0:1776188991.167087:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6487 00000100:00100000:3.0:1776188991.167091:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001d029ec5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664269056:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776188991.167103:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776188991.167138:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001d029ec5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664269056:12345-0@lo:13:osp-pre-1-1.0 Request processed in 46us (541us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188991.167144:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6487 00000100:00100000:3.0:1776188991.167171:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1301 00000100:00100000:3.0:1776188991.167197:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bffe4051 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664268352:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188991.167225:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bffe4051 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664268352:12345-0@lo:400:kworker/u8:0.0 Request processed in 28us (691us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188991.167231:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1301 00000100:00100000:3.0:1776188991.167275:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b51bffd7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664268352:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188991.167286:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000022056a99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664268800:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188991.167297:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008655ce15 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664269056:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776188991.167461:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008db7f874 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664268416:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776188991.167475:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f326e309 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664268864:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776188991.167615:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c1bd7ae7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664268480:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776188991.167658:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188991.167690:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004abaab4f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664268608:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776188991.167710:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188991.167733:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d781d9c8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664268672:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776188991.167795:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188991.167827:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664268672 00000100:00100000:0.0:1776188991.167849:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9045 00000100:00100000:0.0:1776188991.167852:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e71929fa pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664268672:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776188991.167866:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776188991.167899:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e71929fa pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664268672:12345-0@lo:38:kworker/u8:0.0 Request processed in 46us (104us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188991.167905:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9045 00000100:00080000:0.0:1776188991.167934:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776188991.167938:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776188991.167941:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d781d9c8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664268672:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776188991.168103:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664268480 00000100:00100000:2.0:1776188991.168237:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664268608 00000100:00100000:2.0:1776188991.168255:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57137 00000100:00100000:2.0:1776188991.168259:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022af59eb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664268480:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776188991.168267:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776188991.168299:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022af59eb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664268480:12345-0@lo:38:kworker/u8:0.0 Request processed in 39us (641us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188991.168528:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57138 00000100:00100000:1.0:1776188991.168533:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003bb8357b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664268608:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776188991.168541:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776188991.168573:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003bb8357b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664268608:12345-0@lo:38:kworker/u8:0.0 Request processed in 39us (862us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776188991.168579:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57138 00000100:00080000:1.0:1776188991.168624:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776188991.168627:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776188991.168630:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c1bd7ae7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664268480:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776188991.168652:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776188991.168655:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776188991.168657:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004abaab4f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664268608:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776188991.168800:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57137 00010000:02020000:0.0:1776188991.332396:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188991.484403:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188991.588402:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188991.844277:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188992.100897:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188992.356282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188992.509406:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188992.614442:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188992.869120:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188993.125291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188993.381087:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776188993.537814:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188993.636824:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188993.892879:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188994.148298:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188994.404961:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188994.557221:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188994.660367:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188994.918047:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776188994.939945:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188994.940011:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776188994.940046:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654937216 00000100:00100000:0.0:1776188994.940073:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776188994.940085:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654937152 00000100:00100000:3.0:1776188994.940104:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9648 00000100:00100000:3.0:1776188994.940111:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f8f9cfc6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654937216:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1776188994.940124:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1302 00000100:00100000:2.0:1776188994.940130:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002aae1640 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654937152:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1776188994.940211:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f8f9cfc6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654937216:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 99us (274us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188994.940218:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002aae1640 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654937152:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 86us (206us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188994.940222:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9648 00000100:00100000:1.0:1776188994.940223:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654937344 00000100:00100000:2.0:1776188994.940239:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1302 00000100:00100000:1.0:1776188994.940266:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8305 00000100:00100000:1.0:1776188994.940272:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000045a10d8d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2280:x1862459654937344:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1776188994.940303:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000045a10d8d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2280:x1862459654937344:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 31us (233us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188994.940311:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8305 00000100:00100000:1.0:1776188994.942964:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776188994.943097:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654937280 00000100:00100000:0.0:1776188994.943147:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57139 00000100:00100000:0.0:1776188994.943154:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008e777483 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654937280:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:0.0:1776188994.943198:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776188994.943496:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008e777483 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654937280:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 341us (540us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776188994.943505:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57139 00010000:02020000:0.0:1776188995.172822:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776188995.260596:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000036a45663 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664269120:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776188995.260666:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776188995.260729:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776188995.260737:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000b229330 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664269184:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776188995.260761:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188995.260798:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664269120 00000100:00100000:2.0:1776188995.260839:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664269184 00000100:00100000:0.0:1776188995.260854:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6488 00000100:00100000:0.0:1776188995.260861:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cc7962a4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664269120:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776188995.260874:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6489 00000100:00100000:2.0:1776188995.260880:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000030d1626f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664269184:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776188995.260903:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00002000:00000024:2.0:1776188995.260907:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776188995.260951:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cc7962a4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664269120:12345-0@lo:13:osp-pre-0-0.0 Request processed in 89us (285us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188995.260960:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000030d1626f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664269184:12345-0@lo:13:osp-pre-1-0.0 Request processed in 79us (199us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188995.260969:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6489 00000100:00100000:0.0:1776188995.260977:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6488 00000100:00100000:3.0:1776188995.261309:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000b229330 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664269184:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776188995.261339:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000036a45663 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664269120:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776188995.429851:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188995.580823:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188995.685899:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188995.940301:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188996.196512:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:2.0:1776188996.284473:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1776188996.284480:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c094ed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664269312:0@lo:13:osp-pre-0-1.0 00000100:00080000:2.0:1776188996.284480:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776188996.284484:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1776188996.284489:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7c1f715 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664269440:0@lo:400:kworker/u8:1.0 00000100:00080000:2.0:1776188996.284493:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9138 00000100:00080000:2.0:1776188996.284497:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776188996.284516:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776188996.284546:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000114609f6 x1862459664269504/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776188996.284561:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188996.284571:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1776188996.284573:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776188996.284577:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776188996.284580:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776188996.284584:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9138 00000100:00080000:2.0:1776188996.284586:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776188996.284590:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776188996.284599:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1776188996.284603:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000036a45663 x1862459664269632/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776188996.284609:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e718565 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664269248:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776188996.284609:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00080000:2.0:1776188996.284616:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1776188996.284620:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a38efcb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664269376:0@lo:400:kworker/u8:1.0 00000100:00080000:2.0:1776188996.284621:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776188996.284625:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776188996.284630:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9138 00000100:00080000:2.0:1776188996.284631:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776188996.284635:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776188996.284652:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c3059999 x1862459664269696/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776188996.284653:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188996.284668:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188996.284678:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ce461df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664269568:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188996.284698:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000022e717c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664269760:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188996.284710:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188996.284733:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188996.284738:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ebbbbd87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664269888:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188996.284756:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776188996.284764:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003afc110b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664269824:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776188996.284770:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188996.284797:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664269312 00000100:00100000:1.0:1776188996.284800:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776188996.284812:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664269440 00000100:00100000:1.0:1776188996.284830:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000114609f6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664269504:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776188996.284846:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664269248 00000100:00100000:1.0:1776188996.284857:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776188996.284869:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6490 00000100:00100000:1.0:1776188996.284874:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000036a45663 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664269632:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776188996.284878:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000174c978a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664269312:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776188996.284897:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:2.0:1776188996.284902:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776188996.284911:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3059999 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664269696:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776188996.284931:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776188996.284958:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6491 00000100:00100000:1.0:1776188996.284963:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000086fda571 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664269248:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776188996.284976:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776188996.285027:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000174c978a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664269312:12345-0@lo:13:osp-pre-0-1.0 Request processed in 150us (458us total) trans 0 rc 0/0 00000100:00100000:1.0:1776188996.285033:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000086fda571 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664269248:12345-0@lo:13:osp-pre-1-1.0 Request processed in 67us (381us total) trans 0 rc 0/0 00000100:00100000:2.0:1776188996.285038:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6490 00000100:00100000:1.0:1776188996.285045:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6491 00000100:00100000:2.0:1776188996.285067:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664269568 00000100:00100000:1.0:1776188996.285080:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664269376 00000100:00100000:1.0:1776188996.285109:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1303 00000100:00100000:1.0:1776188996.285113:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006f66ddc9 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664269376:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776188996.285144:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006f66ddc9 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664269376:12345-0@lo:400:kworker/u8:1.0 Request processed in 30us (479us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188996.285151:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664269760 00000100:00100000:1.0:1776188996.285164:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1303 00000100:00100000:3.0:1776188996.285198:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664269824 00000100:00100000:0.0:1776188996.285316:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664269888 00000100:00100000:3.0:1776188996.285354:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8306 00000100:00100000:3.0:1776188996.285359:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a87ab8e6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664269760:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188996.285399:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a87ab8e6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664269760:12345-0@lo:400:kworker/u8:1.0 Request processed in 40us (669us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188996.285405:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8306 00000100:00100000:3.0:1776188996.285417:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8307 00000100:00100000:3.0:1776188996.285420:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000994add5b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664269824:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188996.285442:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000994add5b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664269824:12345-0@lo:400:kworker/u8:1.0 Request processed in 21us (646us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188996.285447:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8307 00000100:00100000:3.0:1776188996.285539:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664269504 00000100:00100000:2.0:1776188996.285558:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664269632 00000100:00100000:3.0:1776188996.285560:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9649 00000100:00100000:3.0:1776188996.285566:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008e1f2ed1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664269440:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776188996.285581:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9650 00000100:00100000:2.0:1776188996.285587:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000624e744d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664269568:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188996.285600:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008e1f2ed1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664269440:12345-0@lo:400:kworker/u8:1.0 Request processed in 31us (1040us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188996.285607:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9649 00000100:00100000:3.0:1776188996.285623:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9651 00000100:00100000:2.0:1776188996.285630:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000624e744d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664269568:12345-0@lo:400:kworker/u8:1.0 Request processed in 42us (921us total) trans 0 rc 0/0 00000100:00100000:3.0:1776188996.285637:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000001d1f947 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664269888:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776188996.285641:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9650 00000100:00100000:2.0:1776188996.285660:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57140 00000100:00100000:2.0:1776188996.285664:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000063072b2b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664269504:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776188996.285674:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000001d1f947 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664269888:12345-0@lo:400:kworker/u8:1.0 Request processed in 37us (907us total) trans 0 rc 0/0 00010000:02000000:2.0:1776188996.285678:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776188996.285684:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9651 00000100:00100000:3.0:1776188996.285699:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57141 00000100:00100000:3.0:1776188996.285704:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bde17028 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664269632:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776188996.285713:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776188996.285721:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000063072b2b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664269504:12345-0@lo:38:kworker/u8:1.0 Request processed in 56us (864us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188996.285742:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bde17028 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664269632:12345-0@lo:38:kworker/u8:1.0 Request processed in 37us (846us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776188996.285748:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57141 00000100:00100000:3.0:1776188996.285806:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003afc110b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664269824:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188996.285817:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000022e717c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664269760:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188996.285826:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a38efcb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664269376:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776188996.286560:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57140 00000100:00100000:3.0:1776188996.286561:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c094ed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664269312:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776188996.286623:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664269696 00000100:00100000:2.0:1776188996.286646:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9046 00000100:00100000:2.0:1776188996.286650:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008655ce15 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664269696:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776188996.286660:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776188996.286694:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008655ce15 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664269696:12345-0@lo:38:kworker/u8:1.0 Request processed in 43us (1763us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776188996.286700:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9046 00000100:00100000:3.0:1776188996.286714:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ebbbbd87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664269888:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188996.286737:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e718565 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664269248:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776188996.286748:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ce461df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664269568:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776188996.286757:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7c1f715 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664269440:0@lo:400:kworker/u8:1.0 00000100:00080000:3.0:1776188996.286821:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188996.286825:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188996.286828:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000114609f6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664269504:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776188996.286927:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188996.286930:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188996.286933:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000036a45663 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664269632:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776188996.286942:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776188996.286944:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776188996.286946:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3059999 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664269696:0@lo:38:kworker/u8:1.0 00010000:02020000:0.0:1776188996.455561:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188996.610454:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188996.708345:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188996.964274:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188997.220433:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188997.476283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776188997.629934:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188997.732370:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188997.988243:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188998.244350:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188998.500384:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188998.652770:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188998.756279:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188999.012305:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188999.268283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776188999.524327:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776188999.676837:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776188999.785006:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189000.036288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189000.061706:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189000.061756:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189000.061790:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189000.061859:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189000.061944:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654937408 00000100:00100000:0.0:1776189000.061999:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1304 00000100:00100000:0.0:1776189000.062005:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000002b5a0b3 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654937408:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189000.062068:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000002b5a0b3 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654937408:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 62us (313us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189000.062076:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1304 00000100:00100000:0.0:1776189000.062111:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654937472 00000100:00100000:0.0:1776189000.062134:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654937536 00000100:00100000:0.0:1776189000.062147:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9652 00000100:00100000:0.0:1776189000.062151:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b50afbbd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654937472:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189000.062206:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b50afbbd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654937472:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 54us (508us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189000.062213:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9652 00000100:00100000:0.0:1776189000.062225:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57142 00000100:00100000:0.0:1776189000.062229:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f43285e8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654937536:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776189000.062292:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189000.062333:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f43285e8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654937536:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 104us (475us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189000.062340:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57142 00000100:00100000:3.0:1776189000.062425:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654937600 00000100:00100000:3.0:1776189000.062486:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8308 00000100:00100000:3.0:1776189000.062491:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000516ddb7f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2279:x1862459654937600:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189000.062577:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000516ddb7f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2279:x1862459654937600:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 84us (786us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189000.062584:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8308 00010000:02020000:0.0:1776189000.292295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189000.380604:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776189000.380864:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c094ed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664270016:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189000.380991:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189000.381552:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664270016 00000100:00100000:0.0:1776189000.381612:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6492 00000100:00100000:0.0:1776189000.381618:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000055c776dd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664270016:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189000.381649:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189000.381707:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000055c776dd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664270016:12345-0@lo:13:osp-pre-1-0.0 Request processed in 87us (716us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189000.382184:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000020eb6bf9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664269952:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189000.382371:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189000.382411:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664269952 00000100:00100000:1.0:1776189000.382459:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6493 00000100:00100000:1.0:1776189000.382466:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004d465759 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664269952:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189000.382503:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c094ed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664270016:0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189000.382522:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189000.382573:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004d465759 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664269952:12345-0@lo:13:osp-pre-0-0.0 Request processed in 106us (201us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189000.382581:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6493 00000100:00100000:0.0:1776189000.382605:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6492 00000100:00100000:2.0:1776189000.382683:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000020eb6bf9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664269952:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189000.548365:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189000.702798:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189000.804309:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189001.060260:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189001.316688:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:1.0:1776189001.404405:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189001.404423:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189001.404426:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189001.404434:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9143 00000100:00080000:1.0:1776189001.404439:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00080000:1.0:1776189001.404444:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189001.404452:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e6bd251 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664270080:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189001.404501:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006e718565 x1862459664270208/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189001.404538:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1776189001.404541:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776189001.404542:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189001.404543:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189001.404547:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9143 00000100:00080000:1.0:1776189001.404549:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00080000:1.0:1776189001.404553:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189001.404564:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000023121a7 x1862459664270336/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189001.404572:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189001.404574:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189001.404576:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189001.404578:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9143 00000100:00080000:1.0:1776189001.404581:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 6s 00000100:00080000:1.0:1776189001.404583:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189001.404592:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004abaab4f x1862459664270400/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189001.404638:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d781d9c8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664270528:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189001.404664:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189001.404683:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b51bffd7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664270784:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189001.404704:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189001.404708:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664270528 00000100:00100000:0.0:1776189001.404719:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1776189001.404724:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c1bd7ae7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664270464:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189001.404744:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664270784 00000100:00100000:0.0:1776189001.404746:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189001.404759:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000022056a99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664270720:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189001.404760:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664270464 00000100:00100000:0.0:1776189001.404777:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189001.404777:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8309 00000100:00100000:1.0:1776189001.404782:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000063864031 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664270528:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189001.404796:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664270720 00000100:00100000:0.0:1776189001.404816:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8310 00000100:00100000:0.0:1776189001.404820:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000041c094ed pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664270784:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189001.404889:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8311 00000100:00100000:2.0:1776189001.404894:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b4a5ce06 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664270464:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189001.404931:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000063864031 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664270528:12345-0@lo:400:kworker/u8:1.0 Request processed in 148us (267us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189001.404938:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8309 00000100:00100000:1.0:1776189001.404954:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8312 00000100:00100000:1.0:1776189001.404958:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b5f1e324 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664270720:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189001.404995:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c06060ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664270272:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189001.408101:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000041c094ed pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664270784:12345-0@lo:400:kworker/u8:1.0 Request processed in 3270us (3386us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189001.408115:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8310 00000100:00100000:0.0:1776189001.408242:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b51bffd7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664270784:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189001.408256:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d781d9c8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664270528:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189001.408315:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664270080 00000100:00100000:0.0:1776189001.408356:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1305 00000100:00100000:0.0:1776189001.408361:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000064f5974 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664270080:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189001.408408:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000064f5974 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664270080:12345-0@lo:400:kworker/u8:1.0 Request processed in 46us (3870us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189001.408474:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1305 00000100:00100000:0.0:1776189001.408515:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fd2b3fb0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664270144:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189001.408565:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189001.408594:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000b229330 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664270592:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189001.408620:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189001.408656:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e6bd251 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664270080:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189001.408690:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664270144 00000100:00100000:0.0:1776189001.408732:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9653 00000100:00100000:0.0:1776189001.408736:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0fdb037 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664270144:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189001.408783:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0fdb037 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664270144:12345-0@lo:400:kworker/u8:1.0 Request processed in 46us (219us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189001.408789:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9653 00000100:00100000:0.0:1776189001.408812:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fd2b3fb0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664270144:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189001.408834:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664270592 00000100:00100000:0.0:1776189001.408855:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6494 00000100:00100000:0.0:1776189001.408858:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000154c017b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664270592:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189001.408874:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189001.408904:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000154c017b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664270592:12345-0@lo:13:osp-pre-1-1.0 Request processed in 45us (288us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189001.408910:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6494 00000100:00100000:0.0:1776189001.408938:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000b229330 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664270592:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189001.408994:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e718565 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664270208:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189001.409018:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189001.409032:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000023121a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664270336:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189001.409052:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189001.409063:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004abaab4f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664270400:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189001.409085:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189001.409109:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664270208 00000100:00100000:0.0:1776189001.409132:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664270336 00000100:00100000:0.0:1776189001.409146:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57143 00000100:00100000:0.0:1776189001.409148:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000250cea25 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664270208:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189001.409159:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189001.409211:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000250cea25 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664270208:12345-0@lo:38:kworker/u8:1.0 Request processed in 61us (193us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189001.409217:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57143 00000100:00100000:0.0:1776189001.409228:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57144 00000100:00100000:0.0:1776189001.409231:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001fc79665 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664270336:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189001.409237:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189001.409257:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001fc79665 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664270336:12345-0@lo:38:kworker/u8:1.0 Request processed in 24us (205us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189001.409261:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57144 00000100:00080000:0.0:1776189001.409285:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189001.409288:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189001.409291:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e718565 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664270208:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776189001.409298:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189001.409301:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189001.409303:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000023121a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664270336:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189001.409336:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664270400 00000100:00100000:0.0:1776189001.409353:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9047 00000100:00100000:0.0:1776189001.409357:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000602ee85e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664270400:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189001.409363:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189001.409389:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000602ee85e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664270400:12345-0@lo:38:kworker/u8:1.0 Request processed in 32us (305us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189001.409394:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9047 00000100:00080000:0.0:1776189001.409462:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189001.409465:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189001.409468:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004abaab4f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664270400:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189001.409483:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000058c450e1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664270848:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189001.409501:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189001.409520:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664270848 00000100:00100000:0.0:1776189001.409537:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6495 00000100:00100000:0.0:1776189001.409540:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000023121a7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3363:x1862459664270848:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776189001.409552:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189001.409580:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000023121a7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3363:x1862459664270848:12345-0@lo:13:osp-pre-0-1.0 Request processed in 39us (79us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189001.409585:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6495 00000100:00100000:0.0:1776189001.409610:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000058c450e1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664270848:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189001.418443:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b5f1e324 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664270720:12345-0@lo:400:kworker/u8:1.0 Request processed in 13479us (13661us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189001.418443:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b4a5ce06 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664270464:12345-0@lo:400:kworker/u8:1.0 Request processed in 13542us (13691us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189001.418464:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8312 00000100:00100000:0.0:1776189001.418493:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c1bd7ae7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664270464:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189001.418509:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000022056a99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664270720:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189001.418560:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189001.418749:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008db7f874 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664270656:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189001.418781:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189001.418814:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8311 00000100:00100000:0.0:1776189001.424213:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664270272 00000100:00100000:0.0:1776189001.427394:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664270656 00000100:00100000:0.0:1776189001.427467:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9654 00000100:00100000:0.0:1776189001.427473:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f40104a2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664270272:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189001.427568:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f40104a2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664270272:12345-0@lo:400:kworker/u8:1.0 Request processed in 95us (9164us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189001.427576:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9654 00000100:00100000:0.0:1776189001.427592:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9655 00000100:00100000:0.0:1776189001.427596:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009cc1af34 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664270656:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189001.427624:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009cc1af34 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664270656:12345-0@lo:400:kworker/u8:1.0 Request processed in 27us (8843us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189001.427630:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9655 00000100:00100000:3.0:1776189001.428684:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c06060ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664270272:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189001.428702:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008db7f874 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664270656:0@lo:400:kworker/u8:1.0 00010000:02020000:0.0:1776189001.572271:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189001.724385:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189001.828260:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189002.084342:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189002.340284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189002.596281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189002.748466:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189002.853202:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189003.108282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189003.364301:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189003.621050:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189003.776949:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189003.877391:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189004.132416:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189004.388287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189004.644868:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189004.796500:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189004.900353:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189005.156279:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189005.181729:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189005.181779:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189005.181927:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189005.181965:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189005.182002:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654937728 00000100:00100000:1.0:1776189005.182046:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654937792 00000100:00100000:1.0:1776189005.182065:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9656 00000100:00100000:1.0:1776189005.182072:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ac30dfaa pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654937728:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189005.182143:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ac30dfaa pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654937728:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 70us (422us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189005.182150:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9656 00000100:00100000:1.0:1776189005.182167:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57145 00000100:00100000:1.0:1776189005.182208:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008db7f874 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654937792:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776189005.182223:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189005.182250:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008db7f874 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654937792:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 77us (284us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189005.182253:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654937856 00000100:00100000:1.0:1776189005.182258:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57145 00000100:00100000:2.0:1776189005.182294:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8313 00000100:00100000:2.0:1776189005.182299:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bde4c712 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2278:x1862459654937856:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189005.182346:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bde4c712 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2278:x1862459654937856:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 30us (404us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189005.187309:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8313 00000100:00100000:0.0:1776189005.190295:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654937664 00000100:00100000:0.0:1776189005.190387:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1306 00000100:00100000:0.0:1776189005.190394:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000084b636aa pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654937664:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189005.190481:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000084b636aa pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654937664:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 87us (8702us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189005.190488:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1306 00010000:02020000:0.0:1776189005.416083:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189005.502027:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000020eb6bf9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664270912:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189005.502106:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189005.502276:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664270912 00000100:00100000:2.0:1776189005.502340:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6496 00000100:00100000:2.0:1776189005.502347:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000852f310a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664270912:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189005.502378:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189005.502420:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000852f310a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664270912:12345-0@lo:13:osp-pre-0-0.0 Request processed in 73us (315us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189005.502427:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6496 00000100:00100000:3.0:1776189005.502516:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000020eb6bf9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664270912:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189005.504389:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000058c450e1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664270976:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189005.504441:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189005.508940:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664270976 00000100:00100000:3.0:1776189005.509078:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6497 00000100:00100000:3.0:1776189005.509085:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000266e2268 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664270976:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776189005.509118:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189005.509224:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000266e2268 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664270976:12345-0@lo:13:osp-pre-1-0.0 Request processed in 138us (4783us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189005.509232:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6497 00000100:00100000:0.0:1776189005.510167:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000058c450e1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664270976:0@lo:13:osp-pre-1-0.0 00010000:02020000:1.0:1776189005.668425:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189005.821552:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189005.927453:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189006.180275:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189006.436265:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:1.0:1776189006.524369:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189006.524377:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189006.524380:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189006.524388:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9148 00000100:00080000:1.0:1776189006.524393:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776189006.524399:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189006.524418:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004abaab4f x1862459664271168/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189006.524437:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189006.524440:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189006.524441:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189006.524444:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9148 00000100:00080000:1.0:1776189006.524448:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776189006.524450:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189006.524458:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b51bffd7 x1862459664271296/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189006.524465:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189006.524468:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189006.524469:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189006.524471:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9148 00000100:00080000:1.0:1776189006.524473:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776189006.524476:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189006.524484:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d2d0e609 x1862459664271360/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189006.524589:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c1bd7ae7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664271104:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189006.524649:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189006.524683:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028933edf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664271552:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189006.524703:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189006.524722:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1776189006.524727:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d781d9c8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664271232:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189006.524745:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189006.524756:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d6ea221c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664271616:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189006.524820:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189006.524850:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664271104 00000100:00100000:3.0:1776189006.524883:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664271552 00000100:00100000:3.0:1776189006.524895:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664271232 00000100:00100000:3.0:1776189006.524911:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9657 00000100:00100000:3.0:1776189006.524916:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008b77e02f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664271104:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189006.524958:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008b77e02f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664271104:12345-0@lo:400:kworker/u8:0.0 Request processed in 41us (310us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189006.524964:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9657 00000100:00100000:3.0:1776189006.524978:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9658 00000100:00100000:3.0:1776189006.524981:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c2d64a45 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664271552:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189006.525001:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c2d64a45 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664271552:12345-0@lo:400:kworker/u8:0.0 Request processed in 19us (298us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189006.525006:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9658 00000100:00100000:3.0:1776189006.525015:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9659 00000100:00100000:3.0:1776189006.525018:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000817f7178 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664271232:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189006.525036:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000817f7178 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664271232:12345-0@lo:400:kworker/u8:0.0 Request processed in 17us (290us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189006.525041:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9659 00000100:00100000:3.0:1776189006.525077:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d781d9c8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664271232:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189006.525087:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c1bd7ae7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664271104:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189006.525094:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028933edf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664271552:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189006.525278:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000022056a99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664271040:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189006.525318:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189006.525331:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000037fc1275 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664271488:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189006.525352:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189006.525365:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1776189006.525369:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000020ebaeac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664271424:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189006.525387:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189006.525399:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000def45ccb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664271680:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189006.525416:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189006.525442:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004abaab4f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664271168:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189006.525462:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189006.525472:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b51bffd7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664271296:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189006.525488:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189006.525497:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d2d0e609 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664271360:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189006.525514:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189006.525545:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664271616 00000100:00100000:3.0:1776189006.525565:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664271488 00000100:00100000:3.0:1776189006.525577:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664271424 00000100:00100000:3.0:1776189006.525588:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664271680 00000100:00100000:3.0:1776189006.525602:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8314 00000100:00100000:3.0:1776189006.525605:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000381790a0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664271616:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189006.525636:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000381790a0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664271616:12345-0@lo:400:kworker/u8:0.0 Request processed in 29us (816us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189006.525641:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8314 00000100:00100000:3.0:1776189006.525652:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8315 00000100:00100000:3.0:1776189006.525656:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ad3f4065 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664271488:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189006.525682:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ad3f4065 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664271488:12345-0@lo:400:kworker/u8:0.0 Request processed in 25us (329us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189006.525687:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8315 00000100:00100000:3.0:1776189006.525922:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8316 00000100:00100000:3.0:1776189006.525927:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000064cbacbf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664271424:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189006.525951:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000064cbacbf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664271424:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (563us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189006.525955:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8316 00000100:00100000:3.0:1776189006.525964:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8317 00000100:00100000:3.0:1776189006.525967:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003614bde1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664271680:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189006.525986:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003614bde1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664271680:12345-0@lo:400:kworker/u8:0.0 Request processed in 18us (570us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189006.525990:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8317 00000100:00100000:3.0:1776189006.526015:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000020ebaeac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664271424:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189006.526023:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000def45ccb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664271680:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189006.526030:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000037fc1275 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664271488:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189006.526054:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d6ea221c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664271616:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189006.526088:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664271360 00000100:00100000:3.0:1776189006.526105:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9048 00000100:00100000:3.0:1776189006.526108:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d781d9c8 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664271360:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189006.526123:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189006.526153:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d781d9c8 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664271360:12345-0@lo:38:kworker/u8:0.0 Request processed in 43us (638us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189006.526158:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9048 00000100:00080000:3.0:1776189006.526202:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189006.526205:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189006.526209:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d2d0e609 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664271360:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189006.526316:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664271168 00000100:00100000:1.0:1776189006.526346:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664271296 00000100:00100000:1.0:1776189006.526362:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57146 00000100:00100000:1.0:1776189006.526365:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004f7a6a77 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664271168:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189006.526372:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189006.526404:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004f7a6a77 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664271168:12345-0@lo:38:kworker/u8:0.0 Request processed in 37us (942us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189006.526410:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57146 00000100:00100000:1.0:1776189006.526422:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57147 00000100:00100000:1.0:1776189006.526425:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c1bd7ae7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664271296:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189006.526431:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189006.526450:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c1bd7ae7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664271296:12345-0@lo:38:kworker/u8:0.0 Request processed in 24us (962us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189006.526455:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57147 00000100:00080000:1.0:1776189006.526478:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189006.526481:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189006.526484:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004abaab4f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664271168:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776189006.526492:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189006.526494:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189006.526496:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b51bffd7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664271296:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189006.526848:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776189006.526859:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ae8c4d3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664271744:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189006.526889:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189006.526924:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664271040 00000100:00100000:2.0:1776189006.526951:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1307 00000100:00100000:2.0:1776189006.526954:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c1f77e46 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664271040:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189006.526984:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c1f77e46 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664271040:12345-0@lo:400:kworker/u8:0.0 Request processed in 29us (1666us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189006.526989:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1307 00000100:00100000:2.0:1776189006.527048:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664271744 00000100:00100000:2.0:1776189006.527069:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6498 00000100:00100000:2.0:1776189006.527072:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ee8b6459 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664271744:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189006.527084:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189006.527116:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ee8b6459 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664271744:12345-0@lo:13:osp-pre-1-1.0 Request processed in 42us (226us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189006.527121:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6498 00000100:00100000:2.0:1776189006.527236:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ae8c4d3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664271744:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189006.527614:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000022056a99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664271040:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189006.527749:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776189006.527759:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000797ba66f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664271808:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189006.527786:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189006.527854:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664271808 00000100:00100000:1.0:1776189006.527885:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6499 00000100:00100000:1.0:1776189006.527889:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000058c450e1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664271808:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189006.527900:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189006.527932:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000058c450e1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664271808:12345-0@lo:13:osp-pre-0-1.0 Request processed in 42us (145us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189006.527937:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6499 00000100:00100000:0.0:1776189006.528007:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000797ba66f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664271808:0@lo:13:osp-pre-0-1.0 00010000:02020000:1.0:1776189006.694514:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189006.844443:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189006.948576:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189007.204702:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189007.460934:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189007.717510:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189007.868426:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189008.003599:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189008.260410:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189008.516334:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189008.776477:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189008.892431:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189009.028968:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189009.285590:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189009.540309:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189009.796296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189009.917689:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189010.053081:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189010.299364:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189010.299458:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654937984 00000100:00100000:3.0:1776189010.299522:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9660 00000100:00100000:3.0:1776189010.299530:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f85662d8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654937984:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189010.299609:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f85662d8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654937984:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 78us (253us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189010.299616:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9660 00010000:02020000:1.0:1776189010.308330:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189010.310443:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189010.310494:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189010.310534:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189010.310618:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654938048 00000100:00100000:3.0:1776189010.310677:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57148 00000100:00100000:3.0:1776189010.310683:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008d1771b5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654938048:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:3.0:1776189010.310712:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776189010.310773:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008d1771b5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654938048:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 89us (337us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189010.310792:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57148 00000100:00100000:2.0:1776189010.311147:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654938112 00000100:00100000:2.0:1776189010.311227:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8318 00000100:00100000:2.0:1776189010.311232:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009cace85d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2278:x1862459654938112:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189010.311277:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009cace85d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2278:x1862459654938112:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 44us (743us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189010.311284:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8318 00000100:00100000:0.0:1776189010.311651:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654937920 00000100:00100000:0.0:1776189010.311691:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1308 00000100:00100000:0.0:1776189010.311696:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f055c03f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654937920:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189010.311736:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f055c03f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654937920:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 39us (1242us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189010.311746:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1308 00010000:02020000:1.0:1776189010.564394:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189010.620547:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000576bf6bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664271872:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189010.620621:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189010.620719:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664271872 00000100:00100000:0.0:1776189010.620767:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6500 00000100:00100000:0.0:1776189010.620901:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dabf4d62 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664271872:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189010.620931:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189010.620980:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dabf4d62 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664271872:12345-0@lo:13:osp-pre-1-0.0 Request processed in 206us (360us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189010.620987:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6500 00000100:00100000:2.0:1776189010.621063:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000576bf6bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664271872:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189010.623564:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003de93603 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664271936:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189010.623630:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189010.623900:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664271936 00000100:00100000:2.0:1776189010.623994:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6501 00000100:00100000:2.0:1776189010.623999:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bd5316a5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664271936:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189010.624028:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189010.624075:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bd5316a5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664271936:12345-0@lo:13:osp-pre-0-0.0 Request processed in 75us (446us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189010.624081:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6501 00000100:00100000:0.0:1776189010.624798:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003de93603 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664271936:0@lo:13:osp-pre-0-0.0 00010000:02020000:1.0:1776189010.820295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189010.940997:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189011.076256:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189011.332296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189011.590607:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189011.644532:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012b5e917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664272000:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189011.644598:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776189011.644741:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189011.644747:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189011.644751:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189011.644765:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9153 00000100:00080000:0.0:1776189011.644769:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:0.0:1776189011.644775:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189011.644794:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004be216e7 x1862459664272192/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189011.644836:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189011.644839:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189011.644840:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189011.644844:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9153 00000100:00080000:0.0:1776189011.644846:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00080000:0.0:1776189011.644850:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189011.644861:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b5f1e324 x1862459664272320/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189011.644869:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189011.644872:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189011.644873:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189011.644876:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9153 00000100:00080000:0.0:1776189011.644877:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00080000:0.0:1776189011.644880:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189011.644889:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b4a5ce06 x1862459664272384/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189011.644942:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004be216e7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664272192:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189011.644973:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189011.644991:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5f1e324 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664272320:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189011.645011:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189011.645023:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4a5ce06 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664272384:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189011.645043:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189011.645080:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664272192 00000100:00100000:0.0:1776189011.645110:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664272320 00000100:00100000:0.0:1776189011.645126:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57149 00000100:00100000:0.0:1776189011.645132:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000063864031 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664272192:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189011.645148:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189011.645215:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000063864031 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664272192:12345-0@lo:38:kworker/u8:1.0 Request processed in 83us (242us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189011.645223:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57149 00000100:00100000:0.0:1776189011.645242:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57150 00000100:00100000:0.0:1776189011.645247:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000041c094ed pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664272320:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189011.645254:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189011.645271:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004490682c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664272064:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189011.645275:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000041c094ed pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664272320:12345-0@lo:38:kworker/u8:1.0 Request processed in 28us (264us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189011.645280:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57150 00000100:00100000:0.0:1776189011.645297:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664272384 00000100:00100000:3.0:1776189011.645302:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189011.645315:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9049 00000100:00100000:3.0:1776189011.645318:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bec72d87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664272512:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189011.645319:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000b229330 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664272384:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189011.645325:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189011.645340:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189011.645344:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000b229330 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664272384:12345-0@lo:38:kworker/u8:1.0 Request processed in 25us (301us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189011.645349:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9049 00000100:00100000:3.0:1776189011.645355:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1776189011.645359:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000549bc579 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664272128:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189011.645421:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664272000 00000100:00100000:1.0:1776189011.645460:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6502 00000100:00100000:1.0:1776189011.645465:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003258ba66 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664272000:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189011.645484:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189011.645547:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664272064 00000100:00100000:2.0:1776189011.645576:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1309 00000100:00100000:2.0:1776189011.645579:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000353f317b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664272064:12345-0@lo:400:kworker/u8:1.0 00000100:00080000:0.0:1776189011.646077:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189011.646081:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189011.646085:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004be216e7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664272192:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189011.648979:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189011.649021:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9e630d1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664272576:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189011.649047:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189011.649087:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664272512 00000100:00100000:3.0:1776189011.649127:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8319 00000100:00100000:3.0:1776189011.649132:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000091ef61c6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664272512:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189011.649189:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000091ef61c6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664272512:12345-0@lo:400:kworker/u8:1.0 Request processed in 56us (3848us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189011.649206:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8319 00000100:00100000:3.0:1776189011.649258:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664272128 00000100:00100000:3.0:1776189011.649284:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664272576 00000100:00100000:3.0:1776189011.649300:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9661 00000100:00100000:3.0:1776189011.649303:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000079a89d96 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664272128:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189011.649329:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000079a89d96 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664272128:12345-0@lo:400:kworker/u8:1.0 Request processed in 25us (356us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189011.649334:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9661 00000100:00100000:3.0:1776189011.649343:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9662 00000100:00100000:3.0:1776189011.649346:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000058e0433d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664272576:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189011.649376:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000058e0433d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664272576:12345-0@lo:400:kworker/u8:1.0 Request processed in 17us (317us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189011.649383:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9662 00000100:00100000:3.0:1776189011.649422:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000549bc579 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664272128:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189011.649431:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9e630d1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664272576:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189011.649439:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bec72d87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664272512:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189011.649486:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000576bf6bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664272448:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189011.649508:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189011.649520:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006392c95c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664272704:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189011.649538:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189011.649558:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1776189011.649562:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000075d6a9fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664272256:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189011.649579:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189011.649588:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000045a10d8d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664272640:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189011.649601:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189011.649617:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664272448 00000100:00100000:3.0:1776189011.649645:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664272640 00000100:00100000:3.0:1776189011.649658:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8320 00000100:00100000:3.0:1776189011.649661:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fe93ff8e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664272448:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189011.649688:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fe93ff8e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664272448:12345-0@lo:400:kworker/u8:1.0 Request processed in 26us (180us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189011.649693:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8320 00000100:00100000:3.0:1776189011.649701:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8321 00000100:00100000:3.0:1776189011.650409:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001ae8c4d3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664272640:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189011.650457:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001ae8c4d3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664272640:12345-0@lo:400:kworker/u8:1.0 Request processed in 54us (855us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189011.650463:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8321 00000100:00100000:3.0:1776189011.650501:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664272256 00000100:00100000:3.0:1776189011.650528:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9663 00000100:00100000:3.0:1776189011.650532:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022056a99 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664272256:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189011.650631:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003258ba66 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664272000:12345-0@lo:13:osp-pre-0-1.0 Request processed in 5154us (6022us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189011.650640:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6502 00000100:00100000:1.0:1776189011.650657:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664272704 00000100:00100000:1.0:1776189011.650681:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6503 00000100:00100000:1.0:1776189011.650684:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bde4c712 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664272704:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189011.650697:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189011.650738:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bde4c712 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664272704:12345-0@lo:13:osp-pre-1-1.0 Request processed in 53us (1200us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189011.650745:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6503 00000100:00100000:1.0:1776189011.650835:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000045a10d8d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664272640:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189011.650845:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000576bf6bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664272448:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189011.650858:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006392c95c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664272704:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189011.650869:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012b5e917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664272000:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189011.651080:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022056a99 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664272256:12345-0@lo:400:kworker/u8:1.0 Request processed in 546us (1500us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189011.651087:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9663 00000100:00100000:1.0:1776189011.651471:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000075d6a9fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664272256:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189011.662596:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000353f317b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664272064:12345-0@lo:400:kworker/u8:1.0 Request processed in 17007us (17286us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189011.662625:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1309 00000100:00080000:0.0:1776189011.665907:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189011.665917:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189011.665924:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5f1e324 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664272320:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776189011.665939:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189011.665942:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189011.665946:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4a5ce06 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664272384:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189011.666836:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004490682c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664272064:0@lo:400:kworker/u8:1.0 00010000:02020000:1.0:1776189011.848585:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189011.964373:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189012.102004:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189012.356556:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189012.612290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189012.868465:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189012.988969:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189013.124315:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189013.382529:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189013.636864:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189013.892343:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189014.012395:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189014.148270:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189014.404993:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189014.660681:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189014.916469:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189015.039004:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189015.172344:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189015.428457:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189015.439259:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189015.439305:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189015.439340:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189015.439378:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189015.439415:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654938368 00000100:00100000:3.0:1776189015.439461:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8322 00000100:00100000:3.0:1776189015.439467:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006392c95c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654938368:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1776189015.439528:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006392c95c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654938368:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 60us (150us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189015.439538:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8322 00000100:00100000:1.0:1776189015.439640:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654938240 00000100:00100000:1.0:1776189015.439693:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654938304 00000100:00100000:1.0:1776189015.439712:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9664 00000100:00100000:1.0:1776189015.439716:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b51bffd7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654938240:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1776189015.439746:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b51bffd7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654938240:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 30us (495us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189015.439752:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9664 00000100:00100000:1.0:1776189015.439766:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57151 00000100:00100000:1.0:1776189015.439769:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004abaab4f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654938304:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:1.0:1776189015.439783:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189015.439809:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004abaab4f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654938304:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 39us (505us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189015.439816:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57151 00000100:00100000:2.0:1776189015.440315:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654938176 00000100:00100000:2.0:1776189015.440354:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1310 00000100:00100000:2.0:1776189015.440358:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003c5b81e6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654938176:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1776189015.440394:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003c5b81e6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654938176:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 34us (1054us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189015.440400:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1310 00010000:02020000:1.0:1776189015.684295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189015.740592:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000060073793 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664272768:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189015.740675:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189015.741190:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776189015.741200:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000045a10d8d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664272832:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189015.741239:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189015.741402:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664272768 00000100:00100000:0.0:1776189015.741448:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664272832 00000100:00100000:0.0:1776189015.741484:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6504 00000100:00100000:0.0:1776189015.741489:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004490682c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664272768:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776189015.741518:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189015.741559:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004490682c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664272768:12345-0@lo:13:osp-pre-0-0.0 Request processed in 69us (885us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189015.741566:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6504 00000100:00100000:0.0:1776189015.741583:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6505 00000100:00100000:0.0:1776189015.741586:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b4a5ce06 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664272832:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189015.741602:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189015.741627:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b4a5ce06 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664272832:12345-0@lo:13:osp-pre-1-0.0 Request processed in 39us (389us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189015.741632:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6505 00000100:00100000:3.0:1776189015.741704:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000045a10d8d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664272832:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189015.741717:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000060073793 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664272768:0@lo:13:osp-pre-0-0.0 00010000:02020000:1.0:1776189015.940321:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189016.061089:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189016.196287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189016.452637:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189016.708453:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189016.766964:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012b5e917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664272896:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189016.767025:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189016.767053:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776189016.767058:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bec72d87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664272960:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189016.767098:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189016.767132:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664272960 00000100:00100000:2.0:1776189016.767220:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6506 00000100:00100000:2.0:1776189016.767225:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000094a1a2a8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664272960:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189016.767247:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189016.767285:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000094a1a2a8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664272960:12345-0@lo:13:osp-pre-0-1.0 Request processed in 59us (186us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189016.767291:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6506 00000100:00100000:2.0:1776189016.767343:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bec72d87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664272960:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189016.767381:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664272896 00000100:00100000:2.0:1776189016.767406:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1311 00000100:00100000:2.0:1776189016.767409:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008e1954f2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664272896:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189016.767442:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008e1954f2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664272896:12345-0@lo:400:kworker/u8:0.0 Request processed in 31us (417us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189016.767448:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1311 00000100:00100000:2.0:1776189016.767479:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012b5e917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664272896:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189016.767845:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776189016.767853:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5f1e324 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664273024:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189016.767879:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189016.767903:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664273024 00000100:00100000:3.0:1776189016.767927:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6507 00000100:00100000:3.0:1776189016.767930:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004be216e7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664273024:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189016.767941:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189016.767972:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004be216e7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664273024:12345-0@lo:13:osp-pre-1-1.0 Request processed in 41us (93us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189016.767977:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6507 00000100:00100000:3.0:1776189016.768004:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5f1e324 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664273024:0@lo:13:osp-pre-1-1.0 00000100:00080000:1.0:1776189016.768208:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189016.768214:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189016.768217:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189016.768221:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9158 00000100:00080000:1.0:1776189016.768224:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189016.768228:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189016.768246:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000723457d6 x1862459664273152/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189016.768315:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000073262ad2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664273216:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189016.768343:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189016.768366:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776189016.768372:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024430557 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664273088:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189016.768393:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189016.768416:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664273216 00000100:00100000:2.0:1776189016.768443:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664273088 00000100:00100000:2.0:1776189016.768461:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9665 00000100:00100000:2.0:1776189016.768464:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007d51703e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664273216:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189016.768498:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007d51703e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664273216:12345-0@lo:400:kworker/u8:0.0 Request processed in 33us (155us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189016.768504:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9665 00000100:00100000:2.0:1776189016.768513:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9666 00000100:00100000:2.0:1776189016.768516:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001feee8d8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664273088:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189016.768536:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001feee8d8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664273088:12345-0@lo:400:kworker/u8:0.0 Request processed in 19us (142us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189016.768541:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9666 00000100:00100000:2.0:1776189016.768569:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000073262ad2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664273216:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189016.768584:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024430557 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664273088:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189016.768603:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000723457d6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664273152:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189016.769925:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189016.770020:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664273152 00000100:00100000:3.0:1776189016.770053:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57152 00000100:00100000:3.0:1776189016.770058:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001680f253 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664273152:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189016.770088:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189016.770136:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001680f253 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664273152:12345-0@lo:38:kworker/u8:0.0 Request processed in 78us (216us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189016.770143:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57152 00000100:00080000:3.0:1776189016.770217:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189016.770221:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189016.770225:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000723457d6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664273152:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776189016.774813:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189016.775046:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189016.775050:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189016.775057:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9158 00000100:00080000:1.0:1776189016.775062:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189016.775317:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189016.775359:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cab21754 x1862459664273280/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189016.775384:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189016.775387:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189016.775389:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189016.775392:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9158 00000100:00080000:1.0:1776189016.775394:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189016.775397:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189016.775409:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000fb6b7b14 x1862459664273344/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189016.775460:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000476e266d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664273408:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189016.775527:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189016.775616:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b181f293 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664273472:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189016.775780:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664273408 00000100:00100000:0.0:1776189016.775819:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8323 00000100:00100000:0.0:1776189016.775833:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000981bac76 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664273408:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189016.775902:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189016.775918:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776189016.775922:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005ecd069c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664273536:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189016.775940:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189016.775963:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664273536 00000100:00100000:1.0:1776189016.775982:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9667 00000100:00100000:1.0:1776189016.775985:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000025a2ff5a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664273536:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189016.776024:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000025a2ff5a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664273536:12345-0@lo:400:kworker/u8:0.0 Request processed in 38us (83us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189016.776030:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9667 00000100:00100000:1.0:1776189016.776062:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005ecd069c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664273536:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189016.776160:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664273472 00000100:00100000:3.0:1776189016.776212:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8324 00000100:00100000:3.0:1776189016.776215:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000050c951fb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664273472:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189016.776252:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000050c951fb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664273472:12345-0@lo:400:kworker/u8:0.0 Request processed in 36us (350us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189016.776259:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8324 00000100:00100000:3.0:1776189016.776282:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cab21754 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664273280:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189016.776305:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189016.776318:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fb6b7b14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664273344:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189016.776339:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189016.776364:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664273344 00000100:00100000:3.0:1776189016.776382:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9050 00000100:00100000:3.0:1776189016.776385:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000075d6a9fc pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664273344:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189016.776397:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189016.776427:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000075d6a9fc pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664273344:12345-0@lo:38:kworker/u8:0.0 Request processed in 41us (88us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189016.776433:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9050 00000100:00100000:3.0:1776189016.776452:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664273280 00000100:00100000:3.0:1776189016.776469:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57153 00000100:00100000:3.0:1776189016.776472:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b5f1e324 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664273280:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189016.776479:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189016.776502:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b5f1e324 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664273280:12345-0@lo:38:kworker/u8:0.0 Request processed in 29us (197us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189016.776509:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57153 00000100:00080000:3.0:1776189016.776539:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189016.776542:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189016.776545:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cab21754 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664273280:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776189016.776551:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189016.776553:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189016.776555:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fb6b7b14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664273344:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189016.776580:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b181f293 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664273472:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189016.778974:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000981bac76 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664273408:12345-0@lo:400:kworker/u8:0.0 Request processed in 3142us (3441us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189016.778988:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8323 00000100:00100000:2.0:1776189016.779361:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000476e266d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664273408:0@lo:400:kworker/u8:0.0 00010000:02020000:1.0:1776189016.964355:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189017.084446:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189017.220614:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189017.477259:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189017.732408:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189017.989087:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189018.108529:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189018.244277:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189018.500366:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189018.756290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189019.012294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189019.132461:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189019.268806:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189019.524292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189019.781378:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189020.036284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189020.159425:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189020.292299:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189020.546712:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189020.546771:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189020.546802:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189020.546844:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189020.546858:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654938432 00000100:00100000:3.0:1776189020.546881:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654938496 00000100:00100000:2.0:1776189020.546924:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1312 00000100:00100000:3.0:1776189020.546931:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654938560 00000100:00100000:2.0:1776189020.546933:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001aaffad0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654938432:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1776189020.546954:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9668 00000100:00100000:3.0:1776189020.546961:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000576bf6bf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654938496:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1776189020.547018:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001aaffad0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654938432:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 85us (247us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189020.547025:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1312 00000100:00100000:1.0:1776189020.547066:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654938624 00000100:00100000:2.0:1776189020.547078:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57154 00000100:00100000:2.0:1776189020.547083:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000aa8befb6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654938560:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:2.0:1776189020.547099:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189020.547110:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8325 00000100:00100000:1.0:1776189020.547116:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000549bc579 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654938624:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1776189020.550609:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000576bf6bf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654938496:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 3638us (3896us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189020.550627:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9668 00000100:00100000:2.0:1776189020.554319:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000aa8befb6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654938560:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 7213us (7454us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189020.554338:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57154 00000100:00100000:1.0:1776189020.561468:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000549bc579 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654938624:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 14243us (14557us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189020.561895:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8325 00010000:02020000:3.0:1776189020.563827:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189020.820334:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189020.863091:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776189020.863114:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005ecd069c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664273600:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189020.863220:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189020.863870:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664273600 00000100:00100000:1.0:1776189020.863934:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6508 00000100:00100000:1.0:1776189020.863940:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f9e630d1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664273600:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189020.863969:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189020.864027:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f9e630d1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664273600:12345-0@lo:13:osp-pre-1-0.0 Request processed in 85us (808us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189020.864033:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6508 00000100:00100000:1.0:1776189020.864076:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005ecd069c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664273600:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189020.868674:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004fbf075f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664273664:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189020.868744:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189020.881231:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664273664 00000100:00100000:0.0:1776189020.881302:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6509 00000100:00100000:0.0:1776189020.881309:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005ecd069c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664273664:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776189020.881347:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189020.881428:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005ecd069c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664273664:12345-0@lo:13:osp-pre-0-0.0 Request processed in 119us (12684us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189020.881448:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6509 00000100:00100000:3.0:1776189020.881915:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004fbf075f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664273664:0@lo:13:osp-pre-0-0.0 00010000:02020000:3.0:1776189021.076290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189021.180473:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189021.332414:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189021.588295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189021.844387:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:3.0:1776189021.884756:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189021.884761:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189021.884763:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189021.884770:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9163 00000100:00080000:3.0:1776189021.884772:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189021.884776:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189021.884789:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d97d7d85 x1862459664273856/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189021.884808:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189021.884810:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189021.884811:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189021.884813:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9163 00000100:00080000:3.0:1776189021.884814:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189021.884816:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189021.884820:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000061a1fd5c x1862459664273984/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189021.884825:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189021.884826:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189021.884827:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189021.884828:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9163 00000100:00080000:3.0:1776189021.884830:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189021.884832:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189021.884837:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f6d90a77 x1862459664274048/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189021.885130:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000042c1acfd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664273728:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189021.885219:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189021.885250:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776189021.885257:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007df90c3e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664273792:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189021.885287:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189021.885303:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664273728 00000100:00100000:1.0:1776189021.885306:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bec72d87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664274240:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189021.885336:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189021.885362:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1313 00000100:00100000:0.0:1776189021.885369:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000439c1468 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664273728:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189021.885386:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664274240 00000100:00100000:2.0:1776189021.885415:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6510 00000100:00100000:2.0:1776189021.885423:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000adb1e111 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664274240:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189021.885431:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000439c1468 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664273728:12345-0@lo:400:kworker/u8:1.0 Request processed in 60us (210us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189021.885439:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1313 00002000:00000024:2.0:1776189021.885443:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189021.885481:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000adb1e111 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664274240:12345-0@lo:13:osp-pre-1-1.0 Request processed in 58us (146us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189021.885490:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6510 00000100:00100000:0.0:1776189021.885520:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bec72d87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664274240:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189021.885526:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664273792 00000100:00100000:0.0:1776189021.885535:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000042c1acfd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664273728:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189021.885538:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d97d7d85 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664273856:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189021.885562:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9669 00000100:00100000:2.0:1776189021.885567:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001f4e478a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664273792:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189021.885573:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189021.885592:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000061a1fd5c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664273984:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189021.885610:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001f4e478a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664273792:12345-0@lo:400:kworker/u8:1.0 Request processed in 42us (324us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189021.885617:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9669 00000100:00100000:1.0:1776189021.885622:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189021.885638:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f6d90a77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664274048:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189021.885643:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664273856 00000100:00100000:2.0:1776189021.885665:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664273984 00000100:00100000:2.0:1776189021.885682:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57155 00000100:00100000:2.0:1776189021.885687:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000476e266d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664273856:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189021.885700:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189021.885734:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000476e266d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664273856:12345-0@lo:38:kworker/u8:1.0 Request processed in 47us (166us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189021.885742:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009f0944ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664273920:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189021.885743:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57155 00000100:00100000:1.0:1776189021.885756:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189021.885762:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57156 00000100:00100000:2.0:1776189021.885768:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b181f293 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664273984:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189021.885776:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189021.885781:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776189021.885794:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189021.885800:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189021.885804:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd150e5b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664274176:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189021.885806:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d97d7d85 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664273856:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189021.885811:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b181f293 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664273984:12345-0@lo:38:kworker/u8:1.0 Request processed in 41us (190us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189021.885819:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57156 00000100:00080000:1.0:1776189021.885824:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189021.885828:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189021.885831:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664273920 00000100:00100000:1.0:1776189021.885833:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000061a1fd5c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664273984:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189021.885842:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189021.885852:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9670 00000100:00100000:2.0:1776189021.885856:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004445503c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664273920:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189021.885864:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000045a10d8d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664274304:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189021.885885:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004445503c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664273920:12345-0@lo:400:kworker/u8:1.0 Request processed in 27us (105us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189021.885890:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007c850b41 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664274432:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189021.885895:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9670 00000100:00100000:1.0:1776189021.885902:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189021.885926:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664274304 00000100:00100000:2.0:1776189021.885947:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6511 00000100:00100000:2.0:1776189021.885950:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cab21754 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664274304:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189021.885960:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189021.886134:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664274048 00000100:00100000:1.0:1776189021.886154:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9051 00000100:00100000:1.0:1776189021.886158:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fb6b7b14 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664274048:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189021.886164:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:02000000:1.0:1776189021.886187:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189021.886218:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fb6b7b14 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664274048:12345-0@lo:38:kworker/u8:1.0 Request processed in 59us (465us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189021.886225:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9051 00000100:00080000:1.0:1776189021.886249:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189021.886253:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189021.886257:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f6d90a77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664274048:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189021.886272:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007df90c3e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664273792:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189021.886293:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664274176 00000100:00100000:1.0:1776189021.886321:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664274432 00000100:00100000:1.0:1776189021.886339:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8326 00000100:00100000:1.0:1776189021.886343:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000017cf884a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664274176:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189021.886360:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8327 00000100:00100000:0.0:1776189021.886366:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bec72d87 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+6:3363:x1862459664274432:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189021.887288:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000017cf884a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664274176:12345-0@lo:400:kworker/u8:1.0 Request processed in 942us (1445us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189021.887297:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8326 00000100:00100000:1.0:1776189021.887357:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002dce0820 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664274112:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189021.887363:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bec72d87 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+6:3363:x1862459664274432:12345-0@lo:400:kworker/u8:1.0 Request processed in 994us (1201us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189021.887371:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8327 00000100:00100000:1.0:1776189021.887392:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189021.887411:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007c850b41 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664274432:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189021.887414:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a8591738 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664274368:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189021.887439:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664274112 00000100:00100000:0.0:1776189021.887464:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8328 00000100:00100000:0.0:1776189021.887468:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000723457d6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664274112:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189021.887500:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000723457d6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664274112:12345-0@lo:400:kworker/u8:1.0 Request processed in 31us (110us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189021.887507:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8328 00000100:00100000:1.0:1776189021.887645:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189021.887681:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664274368 00000100:00100000:1.0:1776189021.887688:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002dce0820 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664274112:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189021.887708:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9671 00000100:00100000:0.0:1776189021.887711:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000024430557 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664274368:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189021.887739:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000024430557 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664274368:12345-0@lo:400:kworker/u8:1.0 Request processed in 27us (95us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189021.887743:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9671 00000100:00100000:1.0:1776189021.887771:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a8591738 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664274368:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189021.888124:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd150e5b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664274176:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189021.888151:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009f0944ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664273920:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189021.888252:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cab21754 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664274304:12345-0@lo:13:osp-pre-0-1.0 Request processed in 2295us (2346us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189021.888254:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000045a10d8d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664274304:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189021.888267:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6511 00010000:02020000:3.0:1776189022.100853:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189022.204755:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189022.361081:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189022.612361:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189022.869477:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189023.124285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189023.228352:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189023.381062:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189023.637913:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189023.892457:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189024.148875:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189024.252662:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189024.405778:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189024.661240:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189024.916286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189025.172323:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189025.279969:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189025.428445:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189025.663597:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189025.663643:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189025.663670:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189025.663683:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654938688 00000100:00100000:2.0:1776189025.663740:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1314 00000100:00100000:2.0:1776189025.663747:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006339443e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654938688:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189025.663756:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189025.663816:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006339443e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654938688:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 68us (226us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189025.663824:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1314 00000100:00100000:0.0:1776189025.664053:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654938752 00000100:00100000:1.0:1776189025.664077:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654938816 00000100:00100000:0.0:1776189025.664096:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9672 00000100:00100000:0.0:1776189025.664102:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005788f021 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654938752:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189025.664117:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57157 00000100:00100000:1.0:1776189025.664122:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000069c2fab0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654938816:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00000100:00100000:0.0:1776189025.664139:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005788f021 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654938752:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 34us (495us total) trans 0 rc 0/0 00010000:02000000:1.0:1776189025.664140:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189025.664150:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9672 00000100:00100000:1.0:1776189025.664203:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000069c2fab0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654938816:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 79us (533us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189025.664204:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654938880 00000100:00100000:1.0:1776189025.664211:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57157 00000100:00100000:0.0:1776189025.664249:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8329 00000100:00100000:0.0:1776189025.664254:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000fce6caf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2278:x1862459654938880:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189025.664284:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000fce6caf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2278:x1862459654938880:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 29us (531us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189025.664291:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8329 00010000:02020000:3.0:1776189025.684288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189025.940300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189025.981292:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776189025.981313:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007c9e2a07 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664274496:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189025.981377:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189025.981463:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664274496 00000100:00100000:3.0:1776189025.981526:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6512 00000100:00100000:3.0:1776189025.981531:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000045a10d8d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664274496:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189025.981559:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189025.981612:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000045a10d8d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664274496:12345-0@lo:13:osp-pre-0-0.0 Request processed in 80us (235us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189025.981620:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6512 00000100:00100000:0.0:1776189025.981794:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007c9e2a07 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664274496:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189025.998038:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd150e5b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664274560:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189025.998121:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189025.998549:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664274560 00000100:00100000:0.0:1776189025.998606:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6513 00000100:00100000:0.0:1776189025.998613:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000055462606 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664274560:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189025.998639:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189025.998695:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000055462606 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664274560:12345-0@lo:13:osp-pre-1-0.0 Request processed in 81us (575us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189025.998701:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6513 00000100:00100000:2.0:1776189025.998803:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd150e5b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664274560:0@lo:13:osp-pre-1-0.0 00010000:02020000:3.0:1776189026.197883:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189026.300398:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189026.452360:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189026.708301:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189026.965578:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189027.004737:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776189027.004758:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000073262ad2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664274688:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189027.004826:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189027.004885:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664274688 00000100:00100000:1.0:1776189027.004939:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6514 00000100:00100000:1.0:1776189027.004945:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000012b5e917 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664274688:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189027.004969:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189027.005008:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000012b5e917 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664274688:12345-0@lo:13:osp-pre-1-1.0 Request processed in 62us (183us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189027.005015:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6514 00000100:00100000:1.0:1776189027.005058:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000073262ad2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664274688:0@lo:13:osp-pre-1-1.0 00000100:00080000:1.0:1776189027.005109:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189027.005114:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189027.005117:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189027.005122:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9168 00000100:00080000:1.0:1776189027.005125:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776189027.005128:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189027.005145:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d2d0e609 x1862459664274880/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189027.005209:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189027.005212:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189027.005213:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189027.005218:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9168 00000100:00080000:1.0:1776189027.005220:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776189027.005223:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189027.005235:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000037fc1275 x1862459664275008/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189027.005235:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d2d0e609 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664274880:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776189027.005255:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189027.005258:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189027.005260:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189027.005263:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9168 00000100:00080000:1.0:1776189027.005266:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776189027.005270:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189027.005277:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189027.005284:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000def45ccb x1862459664275072/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189027.005302:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000037fc1275 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664275008:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189027.005342:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189027.005344:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000073262ad2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664274752:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189027.005363:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000def45ccb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664275072:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189027.005378:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189027.005396:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189027.005400:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a680c874 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664275200:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189027.005431:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189027.005443:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664274880 00000100:00100000:1.0:1776189027.005447:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776189027.005453:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000020ebaeac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664275136:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189027.005469:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664275008 00000100:00100000:1.0:1776189027.005475:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189027.005501:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57158 00000100:00100000:1.0:1776189027.005503:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664274752 00000100:00100000:3.0:1776189027.005507:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009610fab3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664274880:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189027.005521:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189027.005528:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1315 00000100:00100000:1.0:1776189027.005533:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a3281d23 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664274752:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189027.005561:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009610fab3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664274880:12345-0@lo:38:kworker/u8:0.0 Request processed in 54us (285us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189027.005568:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57158 00000100:00100000:1.0:1776189027.005581:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a3281d23 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664274752:12345-0@lo:400:kworker/u8:0.0 Request processed in 46us (204us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189027.005586:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57159 00000100:00100000:1.0:1776189027.005592:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1315 00000100:00100000:3.0:1776189027.005594:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c944752e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664275008:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189027.005603:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189027.005628:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c944752e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664275008:12345-0@lo:38:kworker/u8:0.0 Request processed in 36us (287us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189027.005634:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57159 00000100:00100000:3.0:1776189027.005650:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664275072 00000100:00100000:3.0:1776189027.005669:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9052 00000100:00100000:3.0:1776189027.005672:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002fbab827 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664275072:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189027.005678:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189027.005703:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002fbab827 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664275072:12345-0@lo:38:kworker/u8:0.0 Request processed in 30us (309us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189027.005710:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a8591738 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664274624:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189027.005710:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9052 00000100:00080000:3.0:1776189027.005746:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:2.0:1776189027.005748:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1776189027.005752:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189027.005758:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000037fc1275 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664275008:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776189027.005769:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189027.005772:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189027.005774:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d6ea221c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664274944:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189027.005776:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000def45ccb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664275072:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776189027.005789:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189027.005797:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189027.005802:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d2d0e609 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664274880:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189027.005805:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189027.005826:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028933edf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664275328:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189027.005828:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664275200 00000100:00100000:2.0:1776189027.005844:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189027.005871:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664274944 00000100:00100000:3.0:1776189027.005898:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9673 00000100:00100000:3.0:1776189027.005901:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e42367aa pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664274944:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189027.005934:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e42367aa pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664274944:12345-0@lo:400:kworker/u8:0.0 Request processed in 31us (133us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189027.005939:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9673 00000100:00100000:3.0:1776189027.005958:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000797ba66f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664274816:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189027.005980:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189027.005995:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c68f859e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664275264:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189027.006013:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189027.006040:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664274816 00000100:00100000:3.0:1776189027.006061:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664275264 00000100:00100000:3.0:1776189027.006075:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9674 00000100:00100000:3.0:1776189027.006079:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d2d0e609 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664274816:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189027.006111:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d2d0e609 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664274816:12345-0@lo:400:kworker/u8:0.0 Request processed in 32us (131us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189027.006116:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9674 00000100:00100000:3.0:1776189027.006127:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9675 00000100:00100000:3.0:1776189027.006129:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000def45ccb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664275264:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189027.006155:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000def45ccb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664275264:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (141us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189027.006160:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9675 00000100:00100000:3.0:1776189027.006209:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000797ba66f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664274816:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189027.006218:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c68f859e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664275264:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189027.006249:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664274624 00000100:00100000:3.0:1776189027.006271:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6515 00000100:00100000:3.0:1776189027.006275:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ed846a61 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664274624:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776189027.006287:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189027.006314:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ed846a61 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664274624:12345-0@lo:13:osp-pre-0-1.0 Request processed in 38us (568us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189027.006320:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6515 00000100:00100000:2.0:1776189027.009920:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a8591738 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664274624:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189027.009951:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d6ea221c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664274944:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189027.013045:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664275136 00000100:00100000:0.0:1776189027.013119:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664275328 00000100:00100000:0.0:1776189027.013142:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8330 00000100:00100000:0.0:1776189027.013148:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009f0944ec pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664275200:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189027.013242:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8331 00000100:00100000:0.0:1776189027.013250:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009f0944ec pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664275200:12345-0@lo:400:kworker/u8:0.0 Request processed in 100us (7818us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189027.013251:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002dce0820 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664275136:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189027.013263:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8330 00000100:00100000:0.0:1776189027.013281:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8332 00000100:00100000:0.0:1776189027.013286:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ad500dff pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664275328:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189027.013300:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002dce0820 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664275136:12345-0@lo:400:kworker/u8:0.0 Request processed in 50us (7824us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189027.013307:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8331 00000100:00100000:0.0:1776189027.014399:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ad500dff pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664275328:12345-0@lo:400:kworker/u8:0.0 Request processed in 1108us (8550us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189027.014408:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8332 00000100:00100000:3.0:1776189027.014884:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000020ebaeac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664275136:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189027.014899:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000073262ad2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664274752:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189027.014910:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a680c874 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664275200:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189027.014960:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028933edf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664275328:0@lo:400:kworker/u8:0.0 00010000:02020000:3.0:1776189027.220288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189027.324406:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189027.476520:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189027.734272:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189027.988250:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189028.244286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189028.349516:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189028.501478:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189028.757780:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189029.013216:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189029.270113:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189029.372784:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189029.524284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189029.780347:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189030.036286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189030.292499:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189030.397999:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189030.548291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189030.783264:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189030.783313:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189030.783344:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189030.783381:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189030.783449:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654939008 00000100:00100000:3.0:1776189030.783498:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654939072 00000100:00100000:3.0:1776189030.783520:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9676 00000100:00100000:3.0:1776189030.783526:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007444cfc6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654939008:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1776189030.783593:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007444cfc6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654939008:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 66us (336us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189030.783600:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9676 00000100:00100000:3.0:1776189030.783618:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57160 00000100:00100000:3.0:1776189030.783621:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007c9e2a07 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654939072:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:3.0:1776189030.783637:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776189030.783663:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007c9e2a07 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654939072:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 40us (282us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189030.783669:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57160 00000100:00100000:0.0:1776189030.783828:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654939136 00000100:00100000:0.0:1776189030.783861:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8333 00000100:00100000:0.0:1776189030.783865:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000408e2292 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2280:x1862459654939136:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776189030.783897:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000408e2292 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2280:x1862459654939136:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 31us (552us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189030.783903:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8333 00000100:00100000:1.0:1776189030.784933:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654938944 00000100:00100000:1.0:1776189030.784973:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1316 00000100:00100000:1.0:1776189030.784978:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b7303d3d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654938944:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1776189030.785011:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b7303d3d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654938944:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 32us (1699us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189030.785017:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1316 00010000:02020000:3.0:1776189030.806043:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189031.062353:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189031.102909:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007c850b41 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664275392:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189031.102974:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189031.103137:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776189031.103144:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004fbf075f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664275456:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189031.103168:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189031.103252:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664275392 00000100:00100000:2.0:1776189031.103306:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664275456 00000100:00100000:2.0:1776189031.103327:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6516 00000100:00100000:2.0:1776189031.103333:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000028933edf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664275392:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189031.103367:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189031.103412:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000028933edf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664275392:12345-0@lo:13:osp-pre-1-0.0 Request processed in 79us (439us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189031.103419:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6516 00000100:00100000:2.0:1776189031.103436:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6517 00000100:00100000:2.0:1776189031.103439:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a680c874 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664275456:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189031.103455:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189031.103477:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a680c874 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664275456:12345-0@lo:13:osp-pre-0-0.0 Request processed in 38us (310us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189031.103482:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6517 00000100:00100000:3.0:1776189031.103579:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004fbf075f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664275456:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189031.103591:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007c850b41 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664275392:0@lo:13:osp-pre-1-0.0 00010000:02020000:3.0:1776189031.316287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189031.426296:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189031.572388:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189031.828320:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189032.084291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:1.0:1776189032.124346:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189032.124363:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189032.124372:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189032.124379:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9173 00000100:00080000:1.0:1776189032.124383:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00080000:1.0:1776189032.124388:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189032.124413:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007df90c3e x1862459664275712/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189032.124454:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1776189032.124470:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007df90c3e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664275712:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189032.124533:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f6d90a77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664275584:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189032.124542:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776189032.124575:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189032.124577:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189032.124581:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9173 00000100:00080000:1.0:1776189032.124583:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00080000:1.0:1776189032.124588:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189032.124602:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d97d7d85 x1862459664275840/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189032.124612:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189032.124615:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189032.124616:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189032.124621:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9173 00000100:00080000:1.0:1776189032.124624:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 6s 00000100:00080000:1.0:1776189032.124627:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189032.124644:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d63ca2bb x1862459664275968/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189032.124647:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000042c1acfd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664275776:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189032.124702:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007c850b41 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664275904:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189032.131521:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189032.131568:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000060073793 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664275520:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189032.131616:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189032.131645:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005ff20648 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664276032:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189032.131671:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189032.131693:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000694b7c7e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664276288:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189032.131712:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189032.131731:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1776189032.131738:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000061a1fd5c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664275648:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189032.131776:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189032.131791:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e009ccc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664276096:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189032.131814:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189032.131858:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664275584 00000100:00100000:2.0:1776189032.132325:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d97d7d85 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664275840:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189032.132356:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189032.132375:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d63ca2bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664275968:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189032.132398:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189032.132428:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664275712 00000100:00100000:2.0:1776189032.132452:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664275648 00000100:00100000:2.0:1776189032.132474:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664275840 00000100:00100000:2.0:1776189032.132492:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9677 00000100:00100000:2.0:1776189032.132499:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000797ba66f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664275648:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189032.132550:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000797ba66f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664275648:12345-0@lo:400:kworker/u8:0.0 Request processed in 51us (773us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189032.132557:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9677 00000100:00100000:2.0:1776189032.132576:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57161 00000100:00100000:2.0:1776189032.132580:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003de93603 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664275712:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189032.132593:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189032.132624:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003de93603 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664275712:12345-0@lo:38:kworker/u8:0.0 Request processed in 43us (8083us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189032.132630:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57161 00000100:00100000:2.0:1776189032.132640:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57162 00000100:00100000:2.0:1776189032.132643:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009522d095 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664275840:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189032.132651:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189032.132670:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009522d095 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664275840:12345-0@lo:38:kworker/u8:0.0 Request processed in 26us (314us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189032.132675:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57162 00000100:00100000:2.0:1776189032.132695:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664275968 00000100:00100000:2.0:1776189032.132713:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9053 00000100:00100000:2.0:1776189032.132716:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000516ddb7f pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664275968:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189032.132724:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189032.132808:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000516ddb7f pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664275968:12345-0@lo:38:kworker/u8:0.0 Request processed in 90us (411us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189032.132815:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9053 00000100:00100000:2.0:1776189032.132862:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000061a1fd5c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664275648:0@lo:400:kworker/u8:0.0 00000100:00080000:2.0:1776189032.132892:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189032.132896:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189032.132899:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d97d7d85 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664275840:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776189032.132916:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189032.132919:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189032.132921:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d63ca2bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664275968:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776189032.132928:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189032.132930:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189032.132932:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007df90c3e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664275712:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189032.133050:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664275520 00000100:00100000:2.0:1776189032.133077:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6518 00000100:00100000:2.0:1776189032.133081:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000073262ad2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664275520:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189032.133096:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189032.133129:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000073262ad2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664275520:12345-0@lo:13:osp-pre-1-1.0 Request processed in 48us (1513us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189032.133135:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6518 00000100:00100000:2.0:1776189032.133157:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664276032 00000100:00100000:2.0:1776189032.133197:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664276288 00000100:00100000:2.0:1776189032.133212:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664276096 00000100:00100000:2.0:1776189032.133238:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8334 00000100:00100000:2.0:1776189032.133243:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000020ebaeac pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664276032:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189032.133277:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000020ebaeac pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664276032:12345-0@lo:400:kworker/u8:0.0 Request processed in 33us (1606us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189032.133283:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8334 00000100:00100000:2.0:1776189032.133294:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8335 00000100:00100000:2.0:1776189032.133297:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c68f859e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664276288:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189032.133321:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c68f859e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664276288:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (1609us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189032.133326:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8335 00000100:00100000:2.0:1776189032.133336:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8336 00000100:00100000:2.0:1776189032.133339:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000037fc1275 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664276096:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189032.133364:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000037fc1275 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664276096:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (1549us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189032.133369:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8336 00000100:00100000:2.0:1776189032.133393:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e009ccc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664276096:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189032.133403:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005ff20648 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664276032:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189032.133411:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000694b7c7e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664276288:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189032.133424:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000060073793 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664275520:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189032.134041:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189032.134081:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc75cc81 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664276160:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189032.134399:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189032.134436:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664275904 00000100:00100000:1.0:1776189032.134464:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6519 00000100:00100000:1.0:1776189032.134469:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000000115796 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664275904:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189032.134482:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189032.134513:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000000115796 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664275904:12345-0@lo:13:osp-pre-0-1.0 Request processed in 42us (114us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189032.134519:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6519 00000100:00100000:1.0:1776189032.134538:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000002c44196 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664276224:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189032.134565:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189032.134588:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664276224 00000100:00100000:1.0:1776189032.134611:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8337 00000100:00100000:1.0:1776189032.134614:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001f927d70 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664276224:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189032.134641:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001f927d70 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664276224:12345-0@lo:400:kworker/u8:0.0 Request processed in 26us (76us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189032.134647:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8337 00000100:00100000:1.0:1776189032.134674:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007c850b41 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664275904:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189032.134681:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000002c44196 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664276224:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189032.135011:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1317 00000100:00100000:3.0:1776189032.135017:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000089048673 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664275584:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189032.135068:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000089048673 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664275584:12345-0@lo:400:kworker/u8:0.0 Request processed in 51us (3556us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189032.135074:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1317 00000100:00100000:1.0:1776189032.135108:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f6d90a77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664275584:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189032.135158:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664275776 00000100:00100000:2.0:1776189032.135207:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9678 00000100:00100000:2.0:1776189032.135211:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a8591738 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664275776:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189032.135348:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189032.135374:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664276160 00000100:00100000:0.0:1776189032.135397:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9679 00000100:00100000:0.0:1776189032.135401:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cd150e5b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664276160:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189032.135429:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cd150e5b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664276160:12345-0@lo:400:kworker/u8:0.0 Request processed in 27us (82us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189032.135435:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9679 00000100:00100000:0.0:1776189032.135463:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc75cc81 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664276160:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189032.137383:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a8591738 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664275776:12345-0@lo:400:kworker/u8:0.0 Request processed in 2164us (3337us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189032.137396:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9678 00000100:00100000:3.0:1776189032.137686:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000042c1acfd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664275776:0@lo:400:kworker/u8:0.0 00010000:02020000:1.0:1776189032.340302:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189032.444508:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189032.596980:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189032.854833:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189033.108326:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189033.364306:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189033.468499:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189033.620541:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189033.876268:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189034.132285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189034.388483:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189034.492493:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189034.646772:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189034.900276:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189035.156285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189035.412825:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189035.516470:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189035.668926:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189035.901105:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189035.901311:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654939264 00000100:00100000:0.0:1776189035.901374:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9680 00000100:00100000:0.0:1776189035.901380:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000060073793 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654939264:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1776189035.901435:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189035.901457:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000060073793 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654939264:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 75us (359us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189035.901465:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9680 00000100:00100000:0.0:1776189035.901513:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654939200 00000100:00100000:0.0:1776189035.901539:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1318 00000100:00100000:0.0:1776189035.901544:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000007949e4c pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654939200:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1776189035.901575:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189035.901579:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000007949e4c pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654939200:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 33us (145us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189035.901590:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1318 00000100:00100000:2.0:1776189035.901622:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189035.901649:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654939328 00000100:00100000:0.0:1776189035.901685:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57163 00000100:00100000:0.0:1776189035.901689:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c3059999 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654939328:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:0.0:1776189035.901704:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189035.901733:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c3059999 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654939328:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 43us (113us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189035.901740:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57163 00000100:00100000:1.0:1776189035.901996:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654939392 00000100:00100000:1.0:1776189035.902037:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8338 00000100:00100000:1.0:1776189035.902041:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000020eb6bf9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2280:x1862459654939392:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1776189035.902940:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000020eb6bf9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2280:x1862459654939392:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 894us (1364us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189035.902951:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8338 00010000:02020000:1.0:1776189035.924495:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189036.180287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189036.220558:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776189036.220580:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776189036.220581:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ebbbbd87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664276352:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189036.220594:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007c850b41 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664276416:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189036.223513:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189036.223719:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664276416 00000100:00100000:2.0:1776189036.223770:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6520 00000100:00100000:2.0:1776189036.223774:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002bf77dc0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664276416:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189036.223801:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189036.223845:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002bf77dc0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664276416:12345-0@lo:13:osp-pre-1-0.0 Request processed in 69us (339us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189036.223852:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6520 00000100:00100000:0.0:1776189036.223899:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007c850b41 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664276416:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189036.230560:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189036.230723:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664276352 00000100:00100000:3.0:1776189036.230783:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6521 00000100:00100000:3.0:1776189036.230789:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000002c44196 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664276352:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189036.230824:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189036.230883:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000002c44196 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664276352:12345-0@lo:13:osp-pre-0-0.0 Request processed in 94us (485us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189036.230890:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6521 00000100:00100000:1.0:1776189036.231025:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ebbbbd87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664276352:0@lo:13:osp-pre-0-0.0 00010000:02020000:1.0:1776189036.436296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189036.542482:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189036.693700:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189036.948283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189037.204286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189037.244810:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776189037.244830:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000029604218 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664276480:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189037.244890:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189037.244939:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664276480 00000100:00100000:0.0:1776189037.244976:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6522 00000100:00100000:0.0:1776189037.244981:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003eb43bde pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664276480:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776189037.245002:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189037.245037:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003eb43bde pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664276480:12345-0@lo:13:osp-pre-0-1.0 Request processed in 55us (148us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189037.245043:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6522 00000100:00100000:0.0:1776189037.245083:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000029604218 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664276480:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189037.245204:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776189037.245214:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000df4082ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664276544:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189037.245243:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189037.245264:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664276544 00000100:00100000:3.0:1776189037.245335:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6523 00000100:00100000:3.0:1776189037.245338:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000db568034 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664276544:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189037.245348:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189037.245380:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000db568034 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664276544:12345-0@lo:13:osp-pre-1-1.0 Request processed in 41us (139us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189037.245384:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6523 00000100:00100000:0.0:1776189037.246495:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000df4082ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664276544:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189037.248772:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000022e717c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664276608:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189037.248838:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189037.248936:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664276608 00000100:00100000:1.0:1776189037.248991:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1319 00000100:00100000:1.0:1776189037.248997:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c1af929a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664276608:12345-0@lo:400:kworker/u8:0.0 00000100:00080000:2.0:1776189037.249075:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189037.249080:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189037.249082:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189037.249088:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9179 00000100:00080000:2.0:1776189037.249091:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00080000:2.0:1776189037.249095:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189037.249113:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ebbbbd87 x1862459664276736/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189037.249136:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189037.249139:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189037.249140:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189037.249143:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9179 00000100:00080000:2.0:1776189037.249145:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:2.0:1776189037.249149:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189037.249158:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d63ca2bb x1862459664276864/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189037.249165:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189037.249167:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189037.249168:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189037.249171:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9179 00000100:00080000:2.0:1776189037.249190:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 11s 00000100:00080000:2.0:1776189037.249193:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189037.249205:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000042c1acfd x1862459664276928/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189037.249271:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004fbf075f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664276800:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189037.249303:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189037.249325:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776189037.249330:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003afc110b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664276672:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189037.249353:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189037.249378:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664276800 00000100:00100000:3.0:1776189037.249406:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664276672 00000100:00100000:3.0:1776189037.249422:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9681 00000100:00100000:3.0:1776189037.249426:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d6ea221c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664276800:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189037.249461:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d6ea221c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664276800:12345-0@lo:400:kworker/u8:0.0 Request processed in 33us (158us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189037.249468:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9681 00000100:00100000:3.0:1776189037.249483:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9682 00000100:00100000:3.0:1776189037.249486:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001b0eae72 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664276672:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189037.249507:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001b0eae72 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664276672:12345-0@lo:400:kworker/u8:0.0 Request processed in 20us (154us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189037.249512:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9682 00000100:00100000:3.0:1776189037.249559:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003afc110b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664276672:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189037.249569:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004fbf075f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664276800:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189037.249609:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007df90c3e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664276992:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189037.249634:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189037.249675:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ebbbbd87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664276736:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189037.249697:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189037.249709:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d63ca2bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664276864:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189037.249727:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189037.249737:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000042c1acfd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664276928:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189037.249757:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189037.249784:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664276736 00000100:00100000:3.0:1776189037.249802:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664276864 00000100:00100000:3.0:1776189037.249814:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57164 00000100:00100000:3.0:1776189037.249818:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000aa3f364b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664276736:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189037.249831:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189037.249861:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000aa3f364b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664276736:12345-0@lo:38:kworker/u8:0.0 Request processed in 42us (164us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189037.249866:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57164 00000100:00100000:3.0:1776189037.249876:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57165 00000100:00100000:3.0:1776189037.249878:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008573eccf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664276864:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189037.249884:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189037.249901:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008573eccf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664276864:12345-0@lo:38:kworker/u8:0.0 Request processed in 21us (174us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189037.249905:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57165 00000100:00080000:3.0:1776189037.249925:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189037.249928:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189037.249931:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ebbbbd87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664276736:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776189037.249938:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189037.249941:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189037.249943:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d63ca2bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664276864:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189037.249982:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664276928 00000100:00100000:3.0:1776189037.249998:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9054 00000100:00100000:3.0:1776189037.250002:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a829435f pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664276928:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189037.250008:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189037.250032:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a829435f pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664276928:12345-0@lo:38:kworker/u8:0.0 Request processed in 30us (276us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189037.250037:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9054 00000100:00080000:3.0:1776189037.250055:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189037.250058:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189037.250060:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000042c1acfd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664276928:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189037.250110:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664276992 00000100:00100000:3.0:1776189037.250134:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8339 00000100:00100000:3.0:1776189037.250138:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008b54d6c4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664276992:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189037.250168:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008b54d6c4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664276992:12345-0@lo:400:kworker/u8:0.0 Request processed in 29us (534us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189037.250206:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8339 00000100:00100000:3.0:1776189037.250229:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007df90c3e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664276992:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189037.254099:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c1af929a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664276608:12345-0@lo:400:kworker/u8:0.0 Request processed in 5093us (5254us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189037.254115:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1319 00000100:00100000:1.0:1776189037.254377:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007df90c3e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664277120:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189037.254426:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189037.254592:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000022e717c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664276608:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189037.254616:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d97d7d85 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664277056:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189037.254723:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664277120 00000100:00100000:2.0:1776189037.254758:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9683 00000100:00100000:2.0:1776189037.254764:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000df4082ad pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664277120:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189037.254841:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189037.254886:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664277056 00000100:00100000:0.0:1776189037.254929:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8340 00000100:00100000:2.0:1776189037.254932:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000df4082ad pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664277120:12345-0@lo:400:kworker/u8:0.0 Request processed in 165us (504us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189037.254936:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000029604218 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664277056:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189037.254946:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9683 00000100:00100000:0.0:1776189037.254980:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000029604218 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664277056:12345-0@lo:400:kworker/u8:0.0 Request processed in 45us (140us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189037.254987:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8340 00000100:00100000:0.0:1776189037.255016:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d97d7d85 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664277056:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189037.255059:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007df90c3e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664277120:0@lo:400:kworker/u8:0.0 00010000:02020000:1.0:1776189037.460659:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189037.564780:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189037.718269:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189037.972299:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189038.229422:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189038.484366:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189038.590082:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189038.741358:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189038.996286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189039.256559:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189039.508307:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189039.612612:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189039.766643:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189040.020430:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189040.277306:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189040.532538:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189040.636736:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189040.788732:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189041.024789:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189041.024896:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189041.025071:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654939520 00000100:00100000:3.0:1776189041.025134:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9684 00000100:00100000:3.0:1776189041.025142:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cc75cc81 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654939520:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1776189041.025253:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cc75cc81 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654939520:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 111us (358us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189041.025261:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9684 00000100:00100000:2.0:1776189041.031619:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654939456 00000100:00100000:2.0:1776189041.031680:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1320 00000100:00100000:2.0:1776189041.031686:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e2219b01 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654939456:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1776189041.031776:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e2219b01 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654939456:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 89us (6997us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189041.031784:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1320 00000100:00100000:1.0:1776189041.038780:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189041.038829:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189041.038999:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654939584 00000100:00100000:3.0:1776189041.039053:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57166 00000100:00100000:3.0:1776189041.039060:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000694b7c7e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654939584:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:3.0:1776189041.039077:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776189041.039147:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000694b7c7e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654939584:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 86us (376us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189041.039162:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57166 00000100:00100000:0.0:1776189041.039905:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654939648 00000100:00100000:0.0:1776189041.039955:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8341 00000100:00100000:0.0:1776189041.039960:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005ff20648 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2280:x1862459654939648:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776189041.040024:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005ff20648 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2280:x1862459654939648:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 63us (1194us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189041.040030:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8341 00010000:02020000:3.0:1776189041.044874:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189041.301694:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189041.340532:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776189041.340563:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a87ab8e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664277184:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189041.340631:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189041.340809:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002ef57fef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664277248:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189041.340840:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189041.340900:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664277184 00000100:00100000:3.0:1776189041.340938:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664277248 00000100:00100000:3.0:1776189041.340960:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6524 00000100:00100000:3.0:1776189041.340964:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e009ccc0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664277184:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776189041.340991:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189041.341025:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e009ccc0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664277184:12345-0@lo:13:osp-pre-1-0.0 Request processed in 60us (396us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189041.341031:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6524 00000100:00100000:3.0:1776189041.341061:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6525 00000100:00100000:3.0:1776189041.341066:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d63ca2bb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664277248:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189041.341080:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189041.341108:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d63ca2bb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664277248:12345-0@lo:13:osp-pre-0-0.0 Request processed in 42us (268us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189041.341115:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6525 00000100:00100000:3.0:1776189041.341151:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002ef57fef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664277248:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189041.341442:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a87ab8e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664277184:0@lo:13:osp-pre-1-0.0 00010000:02020000:3.0:1776189041.556288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189041.660691:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189041.812516:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189042.069145:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189042.324715:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:2.0:1776189042.364311:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1776189042.364314:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00080000:2.0:1776189042.364318:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189042.364322:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189042.364329:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9184 00000100:00100000:0.0:1776189042.364330:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000994add5b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664277312:0@lo:13:osp-pre-0-1.0 00000100:00080000:2.0:1776189042.364333:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:2.0:1776189042.364338:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189042.364355:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d97d7d85 x1862459664277504/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189042.364376:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189042.364379:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189042.364380:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189042.364384:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9184 00000100:00080000:2.0:1776189042.364385:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00080000:2.0:1776189042.364388:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189042.364392:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189042.364396:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000061a1fd5c x1862459664277632/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189042.364403:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189042.364405:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189042.364406:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189042.364409:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9184 00000100:00080000:2.0:1776189042.364410:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00080000:2.0:1776189042.364413:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189042.364418:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000042c1acfd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664277376:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189042.364424:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ddffdaa5 x1862459664277696/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189042.364489:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189042.364508:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b5045f9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664277760:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189042.364509:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007df90c3e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664277440:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189042.364543:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189042.364543:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189042.364557:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000375d4d18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664277888:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189042.364558:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776189042.364562:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006f1e79e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664277824:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189042.364576:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189042.364579:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189042.364588:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1776189042.364592:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f6d90a77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664277568:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189042.364607:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664277376 00000100:00100000:2.0:1776189042.364611:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189042.364621:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002696d473 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664277952:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189042.364640:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189042.364644:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1321 00000100:00100000:0.0:1776189042.364650:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003e184549 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664277376:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189042.364661:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d97d7d85 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664277504:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189042.364682:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189042.364689:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003e184549 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664277376:12345-0@lo:400:kworker/u8:0.0 Request processed in 38us (200us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189042.364691:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000061a1fd5c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664277632:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189042.364697:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1321 00000100:00100000:2.0:1776189042.364706:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189042.364715:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddffdaa5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664277696:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189042.364736:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189042.364759:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664277312 00000100:00100000:2.0:1776189042.364784:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6526 00000100:00100000:2.0:1776189042.364787:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006e718565 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664277312:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189042.364801:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189042.364827:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006e718565 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664277312:12345-0@lo:13:osp-pre-0-1.0 Request processed in 39us (436us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189042.364832:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6526 00000100:00100000:0.0:1776189042.364844:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664277760 00000100:00100000:2.0:1776189042.364848:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664277824 00000100:00100000:2.0:1776189042.364866:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664277952 00000100:00100000:1.0:1776189042.364968:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664277440 00000100:00100000:1.0:1776189042.364991:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664277888 00000100:00100000:1.0:1776189042.365001:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664277568 00000100:00100000:1.0:1776189042.365008:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664277504 00000100:00100000:1.0:1776189042.365015:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664277632 00000100:00100000:1.0:1776189042.365022:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9685 00000100:00100000:1.0:1776189042.365037:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000880386e2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664277440:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189042.365445:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000880386e2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664277440:12345-0@lo:400:kworker/u8:0.0 Request processed in 417us (901us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189042.365453:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9685 00000100:00100000:1.0:1776189042.365469:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9686 00000100:00100000:1.0:1776189042.365472:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a969b6e4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664277888:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189042.365498:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a969b6e4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664277888:12345-0@lo:400:kworker/u8:0.0 Request processed in 25us (922us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189042.365503:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9686 00000100:00100000:1.0:1776189042.365514:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9687 00000100:00100000:1.0:1776189042.365517:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c921b786 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664277568:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189042.365578:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c921b786 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664277568:12345-0@lo:400:kworker/u8:0.0 Request processed in 60us (966us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189042.365585:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9687 00000100:00100000:1.0:1776189042.365597:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57167 00000100:00100000:1.0:1776189042.365600:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c731e56f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664277504:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189042.365612:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189042.365644:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c731e56f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664277504:12345-0@lo:38:kworker/u8:0.0 Request processed in 43us (962us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189042.365650:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57167 00000100:00100000:1.0:1776189042.365661:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57168 00000100:00100000:1.0:1776189042.365664:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b3b3d75f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664277632:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189042.365672:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189042.365693:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b3b3d75f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664277632:12345-0@lo:38:kworker/u8:0.0 Request processed in 28us (986us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189042.365699:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57168 00000100:00080000:1.0:1776189042.365740:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189042.365744:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189042.365747:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d97d7d85 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664277504:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776189042.365761:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189042.365764:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189042.365767:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000061a1fd5c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664277632:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189042.365805:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f6d90a77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664277568:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189042.365815:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007df90c3e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664277440:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189042.365824:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000375d4d18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664277888:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189042.365865:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776189042.365875:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004afd2049 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664278016:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189042.365903:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189042.365966:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000994add5b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664277312:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189042.365978:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000042c1acfd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664277376:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189042.366042:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664277696 00000100:00100000:1.0:1776189042.366064:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9055 00000100:00100000:1.0:1776189042.366068:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f34aeb18 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664277696:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189042.366075:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189042.366398:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f34aeb18 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664277696:12345-0@lo:38:kworker/u8:0.0 Request processed in 328us (1661us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189042.366405:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9055 00000100:00080000:1.0:1776189042.366440:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189042.366444:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189042.366447:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddffdaa5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664277696:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189042.368558:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8342 00000100:00100000:0.0:1776189042.368570:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008e6bd251 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664277824:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189042.368649:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008e6bd251 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664277824:12345-0@lo:400:kworker/u8:0.0 Request processed in 79us (4068us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189042.368655:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8342 00000100:00100000:0.0:1776189042.368673:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8343 00000100:00100000:0.0:1776189042.368677:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fd2b3fb0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664277760:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189042.368706:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fd2b3fb0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664277760:12345-0@lo:400:kworker/u8:0.0 Request processed in 28us (4162us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189042.368711:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8343 00000100:00100000:0.0:1776189042.368722:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8344 00000100:00100000:0.0:1776189042.368725:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001b0d11fc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664277952:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189042.368756:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001b0d11fc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664277952:12345-0@lo:400:kworker/u8:0.0 Request processed in 30us (4116us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189042.368762:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8344 00000100:00100000:0.0:1776189042.368806:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006f1e79e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664277824:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189042.368817:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b5045f9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664277760:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189042.368855:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002696d473 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664277952:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189042.368893:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664278016 00000100:00100000:0.0:1776189042.368935:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6527 00000100:00100000:0.0:1776189042.368943:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d50ac393 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664278016:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189042.368956:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189042.368991:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d50ac393 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664278016:12345-0@lo:13:osp-pre-1-1.0 Request processed in 47us (3088us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189042.368997:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6527 00000100:00100000:0.0:1776189042.369025:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004afd2049 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664278016:0@lo:13:osp-pre-1-1.0 00010000:02020000:3.0:1776189042.580580:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189042.684385:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189042.836274:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189043.093563:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189043.348281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189043.604408:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189043.708732:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189043.860727:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189044.116327:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189044.373672:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189044.628287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189044.732409:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189044.885291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189045.140295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189045.396269:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189045.652305:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189045.757112:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189045.909275:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189046.143873:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189046.143997:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189046.144028:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189046.144070:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189046.144222:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654939904 00000100:00100000:0.0:1776189046.144272:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8345 00000100:00100000:0.0:1776189046.144280:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ad3c7e22 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654939904:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776189046.144356:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ad3c7e22 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654939904:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 76us (327us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189046.144363:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8345 00000100:00100000:1.0:1776189046.144668:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654939840 00000100:00100000:1.0:1776189046.144743:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654939776 00000100:00100000:2.0:1776189046.149468:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654939712 00000100:00100000:2.0:1776189046.149539:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1322 00000100:00100000:2.0:1776189046.149546:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000089ddaa97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654939712:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1776189046.149620:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000089ddaa97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654939712:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 74us (5755us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189046.149628:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1322 00000100:00100000:0.0:1776189046.150732:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9688 00000100:00100000:0.0:1776189046.150742:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fa770ed8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654939776:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776189046.150809:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fa770ed8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654939776:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 67us (6739us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189046.150830:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9688 00000100:00100000:0.0:1776189046.150853:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57169 00000100:00100000:0.0:1776189046.150857:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b3632b78 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654939840:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:0.0:1776189046.150871:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189046.150897:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b3632b78 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654939840:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 39us (6901us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189046.150903:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57169 00010000:02020000:3.0:1776189046.164286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189046.421808:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189046.460495:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002ef57fef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664278080:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189046.460591:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189046.460619:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c8b239a6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664278144:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189046.460656:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189046.468033:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664278080 00000100:00100000:0.0:1776189046.468107:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664278144 00000100:00100000:0.0:1776189046.468139:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6528 00000100:00100000:0.0:1776189046.468145:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000994add5b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664278080:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776189046.468193:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189046.468270:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000994add5b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664278080:12345-0@lo:13:osp-pre-0-0.0 Request processed in 125us (7681us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189046.468278:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6528 00000100:00100000:0.0:1776189046.468293:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6529 00000100:00100000:0.0:1776189046.468296:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004afd2049 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664278144:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189046.468312:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189046.468344:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004afd2049 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664278144:12345-0@lo:13:osp-pre-1-0.0 Request processed in 46us (7689us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189046.468350:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6529 00000100:00100000:0.0:1776189046.468400:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c8b239a6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664278144:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189046.468582:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002ef57fef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664278080:0@lo:13:osp-pre-0-0.0 00010000:02020000:3.0:1776189046.676374:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189046.780384:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189046.932274:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189047.189139:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189047.444658:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189047.484748:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00080000:0.0:1776189047.484769:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1776189047.484769:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002696d473 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664278208:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189047.484832:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776189047.484838:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189047.484842:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189047.484848:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9189 00000100:00080000:0.0:1776189047.484852:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1776189047.484858:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189047.484877:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a87ab8e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664278272:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189047.484888:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000b36a9c1 x1862459664278400/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189047.484908:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189047.484924:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776189047.484929:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007c850b41 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664278336:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189047.484951:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189047.484993:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664278336 00000100:00100000:1.0:1776189047.485038:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9689 00000100:00080000:0.0:1776189047.485065:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189047.485068:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189047.485070:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189047.485074:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9189 00000100:00080000:0.0:1776189047.485076:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1776189047.485079:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189047.485092:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006cdcc0ce x1862459664278528/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189047.485100:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189047.485103:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189047.485104:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189047.485107:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9189 00000100:00080000:0.0:1776189047.485109:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1776189047.485111:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189047.485122:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f12977ad x1862459664278592/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189047.485160:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006f1e79e6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664278336:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189047.485170:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000b36a9c1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664278400:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189047.485223:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189047.485233:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006f1e79e6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664278336:12345-0@lo:400:kworker/u8:1.0 Request processed in 187us (281us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189047.485241:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9689 00000100:00100000:0.0:1776189047.485245:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006cdcc0ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664278528:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189047.485259:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664278400 00000100:00100000:0.0:1776189047.485271:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189047.485343:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664278208 00000100:00100000:2.0:1776189047.485349:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ff579061 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664278656:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189047.485381:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189047.485385:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6530 00000100:00100000:3.0:1776189047.485391:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009b5045f9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664278208:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189047.485400:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004ac5ba05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664278912:0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189047.485409:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189047.485434:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189047.485528:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664278528 00000100:00100000:1.0:1776189047.485552:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57170 00000100:00100000:1.0:1776189047.485556:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000051d6dabb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664278400:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189047.485570:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189047.485585:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:2.0:1776189047.485593:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002e9e0e35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664278464:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189047.485604:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000051d6dabb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664278400:12345-0@lo:38:kworker/u8:1.0 Request processed in 46us (381us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189047.485611:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57170 00000100:00100000:2.0:1776189047.485625:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189047.485627:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57171 00000100:00100000:1.0:1776189047.485632:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c5eb288 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664278528:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189047.485639:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002ea466a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664278848:0@lo:400:kworker/u8:1.0 00010000:02000000:1.0:1776189047.485641:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189047.485666:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189047.485677:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c5eb288 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664278528:12345-0@lo:38:kworker/u8:1.0 Request processed in 42us (406us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189047.485685:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57171 00000100:00100000:2.0:1776189047.485697:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664278912 00000100:00100000:1.0:1776189047.485700:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664278464 00000100:00100000:2.0:1776189047.485720:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6531 00000100:00100000:1.0:1776189047.485725:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9690 00000100:00100000:2.0:1776189047.485727:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c510b01d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664278912:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189047.485733:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001a690fbd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664278464:12345-0@lo:400:kworker/u8:1.0 00002000:00000024:2.0:1776189047.485740:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189047.485777:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001a690fbd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664278464:12345-0@lo:400:kworker/u8:1.0 Request processed in 45us (154us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189047.485784:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9690 00000100:00100000:1.0:1776189047.485828:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039f4c6c3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664278720:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189047.486517:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f12977ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664278592:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189047.486567:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c510b01d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664278912:12345-0@lo:13:osp-pre-1-1.0 Request processed in 840us (1133us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189047.486574:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6531 00000100:00100000:2.0:1776189047.486604:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664278272 00000100:00100000:2.0:1776189047.486631:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1323 00000100:00100000:2.0:1776189047.486635:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006e7c9c41 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664278272:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189047.486804:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189047.486841:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007c850b41 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664278336:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189047.486863:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776189047.486868:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fc3b4133 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664278784:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189047.486890:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189047.486917:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664278784 00000100:00100000:1.0:1776189047.486942:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9691 00000100:00100000:1.0:1776189047.486946:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000375d4d18 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664278784:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189047.486979:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000375d4d18 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664278784:12345-0@lo:400:kworker/u8:1.0 Request processed in 32us (88us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189047.486984:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9691 00000100:00100000:1.0:1776189047.487010:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fc3b4133 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664278784:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189047.487167:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664278656 00000100:00100000:1.0:1776189047.487210:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664278848 00000100:00100000:1.0:1776189047.487224:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664278720 00000100:00100000:1.0:1776189047.487240:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8346 00000100:00100000:1.0:1776189047.487244:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001fe1325f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664278656:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189047.487274:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001fe1325f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664278656:12345-0@lo:400:kworker/u8:1.0 Request processed in 29us (1893us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189047.487279:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8346 00000100:00100000:1.0:1776189047.487290:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8347 00000100:00100000:1.0:1776189047.487293:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000048c7c2e3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664278848:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189047.487315:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000048c7c2e3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664278848:12345-0@lo:400:kworker/u8:1.0 Request processed in 21us (1651us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189047.487320:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8347 00000100:00100000:1.0:1776189047.487328:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8348 00000100:00100000:1.0:1776189047.487331:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ddffdaa5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664278720:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189047.487354:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ddffdaa5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664278720:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (551us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189047.487358:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8348 00000100:00100000:1.0:1776189047.487378:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039f4c6c3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664278720:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189047.487521:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002e9e0e35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664278464:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189047.487534:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002ea466a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664278848:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189047.487542:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ff579061 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664278656:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189047.487554:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004ac5ba05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664278912:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189047.491522:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009b5045f9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664278208:12345-0@lo:13:osp-pre-0-1.0 Request processed in 6123us (6683us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189047.491540:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6530 00000100:00100000:1.0:1776189047.492085:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002696d473 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664278208:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189047.492592:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776189047.492643:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189047.492647:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189047.492651:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000b36a9c1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664278400:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776189047.492663:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189047.492665:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189047.492668:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006cdcc0ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664278528:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189047.493096:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664278592 00000100:00100000:2.0:1776189047.493252:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006e7c9c41 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664278272:12345-0@lo:400:kworker/u8:1.0 Request processed in 6613us (8342us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189047.493263:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1323 00000100:00100000:3.0:1776189047.493352:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9056 00000100:00100000:3.0:1776189047.493358:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006b6d51e7 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664278592:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189047.493374:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189047.493429:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006b6d51e7 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664278592:12345-0@lo:38:kworker/u8:1.0 Request processed in 71us (839us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189047.493436:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9056 00000100:00100000:0.0:1776189047.493500:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a87ab8e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664278272:0@lo:400:kworker/u8:1.0 00000100:00080000:2.0:1776189047.493566:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189047.493570:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189047.493573:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f12977ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664278592:0@lo:38:kworker/u8:1.0 00010000:02020000:3.0:1776189047.700640:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189047.807445:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189047.957063:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189048.212325:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189048.471274:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189048.724296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189048.828434:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189048.981050:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189049.236547:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189049.494798:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189049.748332:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189049.852725:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189050.005418:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189050.261804:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189050.516295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189050.772384:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189050.876588:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189051.028477:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189051.261391:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189051.261436:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189051.261492:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189051.261523:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654940096 00000100:00100000:0.0:1776189051.261579:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654939968 00000100:00100000:3.0:1776189051.261655:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654940032 00000100:00100000:0.0:1776189051.261670:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1324 00000100:00100000:0.0:1776189051.261678:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d3d96460 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654939968:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189051.261750:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d3d96460 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654939968:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 72us (258us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189051.261758:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1324 00000100:00100000:1.0:1776189051.264044:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9692 00000100:00100000:1.0:1776189051.264056:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000b36a9c1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654940032:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189051.267780:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000b36a9c1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654940032:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 3715us (6335us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189051.267798:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9692 00000100:00100000:1.0:1776189051.267850:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57172 00000100:00100000:1.0:1776189051.267857:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002696d473 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654940096:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776189051.267875:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189051.267945:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002696d473 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654940096:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 88us (6562us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189051.267952:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57172 00000100:00100000:0.0:1776189051.271029:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189051.271073:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654940160 00000100:00100000:2.0:1776189051.271193:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8349 00000100:00100000:2.0:1776189051.271202:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006cdcc0ce pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2280:x1862459654940160:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189051.271281:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006cdcc0ce pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2280:x1862459654940160:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 79us (260us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189051.271289:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8349 00010000:02020000:3.0:1776189051.285071:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189051.540307:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189051.580486:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776189051.580507:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000005ed8ead pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664278976:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189051.580572:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189051.580658:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664278976 00000100:00100000:2.0:1776189051.580712:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6532 00000100:00100000:2.0:1776189051.580719:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002e9e0e35 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664278976:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189051.580750:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189051.580765:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002ef57fef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664279040:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189051.580801:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002e9e0e35 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664278976:12345-0@lo:13:osp-pre-1-0.0 Request processed in 82us (229us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189051.580807:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189051.580813:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6532 00000100:00100000:2.0:1776189051.580833:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664279040 00000100:00100000:3.0:1776189051.580851:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000005ed8ead pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664278976:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189051.580857:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6533 00000100:00100000:2.0:1776189051.580861:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022e717c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664279040:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189051.580877:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189051.580908:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022e717c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664279040:12345-0@lo:13:osp-pre-0-0.0 Request processed in 46us (106us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189051.580913:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6533 00000100:00100000:3.0:1776189051.580968:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002ef57fef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664279040:0@lo:13:osp-pre-0-0.0 00010000:02020000:3.0:1776189051.796471:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189051.907938:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189052.058600:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189052.316288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189052.572274:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189052.604576:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b39ea587 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664279168:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189052.604659:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189052.604695:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776189052.604702:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007c850b41 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664279104:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189052.604726:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1776189052.604742:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189052.604748:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189052.604751:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1776189052.604756:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003189c953 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664279232:0@lo:400:kworker/u8:0.0 00000100:00080000:2.0:1776189052.604773:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9194 00000100:00080000:2.0:1776189052.604776:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776189052.604782:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189052.604787:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189052.604803:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776189052.604806:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005ee0b9e5 x1862459664279360/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189052.604809:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001a975e34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664279296:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189052.604841:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1776189052.604847:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189052.604852:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189052.604853:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189052.604858:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9194 00000100:00080000:2.0:1776189052.604860:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776189052.604864:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189052.604873:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664279168 00000100:00100000:2.0:1776189052.604877:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000005ed8ead x1862459664279488/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189052.604885:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189052.604888:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189052.604890:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189052.604892:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9194 00000100:00080000:2.0:1776189052.604894:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776189052.604897:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189052.604908:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004ac5ba05 x1862459664279552/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189052.604925:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664279104 00000100:00100000:0.0:1776189052.604944:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6534 00000100:00100000:0.0:1776189052.604951:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c9476df4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664279168:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189052.604963:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005ee0b9e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664279360:0@lo:38:kworker/u8:0.0 00002000:00000024:0.0:1776189052.604971:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189052.604996:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189052.605015:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000005ed8ead pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664279488:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189052.605020:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c9476df4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664279168:12345-0@lo:13:osp-pre-1-1.0 Request processed in 66us (359us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189052.605028:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6534 00000100:00100000:2.0:1776189052.605042:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189052.605047:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6535 00000100:00100000:0.0:1776189052.605052:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cb00f0cd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664279104:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189052.605059:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004ac5ba05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664279552:0@lo:38:kworker/u8:0.0 00002000:00000024:0.0:1776189052.605062:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189052.605091:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189052.605097:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cb00f0cd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664279104:12345-0@lo:13:osp-pre-0-1.0 Request processed in 42us (369us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189052.605119:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664279296 00000100:00100000:2.0:1776189052.605142:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664279360 00000100:00100000:2.0:1776189052.605152:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664279488 00000100:00100000:2.0:1776189052.605164:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9693 00000100:00100000:2.0:1776189052.605168:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000080ecc908 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664279296:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189052.605220:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000080ecc908 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664279296:12345-0@lo:400:kworker/u8:0.0 Request processed in 51us (381us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189052.605226:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9693 00000100:00100000:2.0:1776189052.605238:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57173 00000100:00100000:2.0:1776189052.605241:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000013bfaae8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664279360:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189052.605255:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189052.605266:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6535 00000100:00100000:2.0:1776189052.605284:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000013bfaae8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664279360:12345-0@lo:38:kworker/u8:0.0 Request processed in 40us (288us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189052.605291:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57173 00000100:00100000:0.0:1776189052.605295:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664279232 00000100:00100000:2.0:1776189052.605314:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57174 00000100:00100000:2.0:1776189052.605319:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000affce7e0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664279488:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189052.605326:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189052.605334:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1325 00000100:00100000:0.0:1776189052.605339:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000027854250 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664279232:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189052.605349:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000affce7e0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664279488:12345-0@lo:38:kworker/u8:0.0 Request processed in 29us (308us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189052.605356:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57174 00000100:00100000:1.0:1776189052.605450:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aafe5128 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664279680:0@lo:400:kworker/u8:0.0 00000100:00080000:2.0:1776189052.605480:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189052.605484:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189052.605487:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005ee0b9e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664279360:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189052.605544:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000023a5b7b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664279424:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189052.613675:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000027854250 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664279232:12345-0@lo:400:kworker/u8:0.0 Request processed in 8327us (8881us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189052.613695:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1325 00000100:00100000:0.0:1776189052.613819:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009e6a2c02 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664279744:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189052.613868:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189052.613916:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664279744 00000100:00100000:0.0:1776189052.613954:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9694 00000100:00100000:0.0:1776189052.613959:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000099f8c850 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664279744:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189052.614007:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000099f8c850 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664279744:12345-0@lo:400:kworker/u8:0.0 Request processed in 47us (139us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189052.614012:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9694 00000100:00100000:0.0:1776189052.614041:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009e6a2c02 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664279744:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189052.614079:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c8b239a6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664279616:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189052.614104:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189052.614134:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007c850b41 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664279104:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189052.614148:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b39ea587 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664279168:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189052.614202:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664279616 00000100:00100000:0.0:1776189052.614233:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8350 00000100:00100000:0.0:1776189052.614237:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd47b990 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664279616:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189052.614268:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd47b990 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664279616:12345-0@lo:400:kworker/u8:0.0 Request processed in 31us (164us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189052.614274:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8350 00000100:00100000:0.0:1776189052.614294:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c8b239a6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664279616:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189052.614330:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664279552 00000100:00100000:0.0:1776189052.614347:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9057 00000100:00100000:0.0:1776189052.614351:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001b6b6588 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664279552:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189052.614364:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189052.614391:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001b6b6588 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664279552:12345-0@lo:38:kworker/u8:0.0 Request processed in 40us (9303us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189052.614397:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9057 00000100:00100000:1.0:1776189052.618031:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189052.618104:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001a975e34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664279296:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189052.618120:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003189c953 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664279232:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189052.618214:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664279680 00000100:00100000:0.0:1776189052.618267:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8351 00000100:00100000:0.0:1776189052.618272:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007c850b41 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664279680:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189052.618334:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007c850b41 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664279680:12345-0@lo:400:kworker/u8:0.0 Request processed in 60us (310us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189052.618342:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8351 00000100:00100000:1.0:1776189052.618422:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aafe5128 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664279680:0@lo:400:kworker/u8:0.0 00000100:00080000:2.0:1776189052.625277:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189052.625288:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189052.625295:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000005ed8ead pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664279488:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776189052.625318:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189052.625321:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189052.625324:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004ac5ba05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664279552:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189052.626632:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189052.626762:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664279424 00000100:00100000:0.0:1776189052.626818:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9695 00000100:00100000:0.0:1776189052.626824:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ebbbbd87 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664279424:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189052.626882:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ebbbbd87 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664279424:12345-0@lo:400:kworker/u8:0.0 Request processed in 57us (256us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189052.626888:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9695 00000100:00100000:3.0:1776189052.627023:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000023a5b7b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664279424:0@lo:400:kworker/u8:0.0 00010000:02020000:3.0:1776189052.828297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189052.924422:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189053.084359:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189053.340328:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189053.597410:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189053.852292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189053.950372:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189054.109898:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189054.364298:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189054.620742:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189054.876297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189054.974903:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189055.132517:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189055.388305:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189055.644293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189055.901145:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189055.996413:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189056.156308:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189056.383488:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189056.383545:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189056.383579:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189056.383627:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189056.383761:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654940416 00000100:00100000:0.0:1776189056.383826:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654940224 00000100:00100000:3.0:1776189056.383827:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8352 00000100:00100000:3.0:1776189056.383835:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000061a1fd5c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2279:x1862459654940416:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189056.383863:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1326 00000100:00100000:0.0:1776189056.383870:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000192d1549 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654940224:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189056.383903:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000061a1fd5c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2279:x1862459654940416:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 67us (324us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189056.383909:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000192d1549 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654940224:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 36us (361us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189056.383917:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8352 00000100:00100000:0.0:1776189056.383920:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1326 00000100:00100000:3.0:1776189056.383975:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654940288 00000100:00100000:3.0:1776189056.384003:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654940352 00000100:00100000:3.0:1776189056.384020:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9696 00000100:00100000:3.0:1776189056.384024:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f6d90a77 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654940288:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189056.384069:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57175 00000100:00100000:3.0:1776189056.384069:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f6d90a77 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654940288:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 38us (584us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189056.384077:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000042c1acfd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654940352:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00000100:00100000:3.0:1776189056.384081:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9696 00010000:02000000:0.0:1776189056.384098:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189056.384125:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000042c1acfd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654940352:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 49us (499us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189056.384132:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57175 00010000:02020000:1.0:1776189056.412292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189056.668299:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189056.701118:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000046fee2b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664279872:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189056.701150:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004fbf075f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664279808:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189056.708293:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189056.709076:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664279872 00000100:00100000:3.0:1776189056.709496:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6536 00000100:00100000:3.0:1776189056.709505:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cae4b12b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664279872:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776189056.709548:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189056.709626:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cae4b12b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664279872:12345-0@lo:13:osp-pre-1-0.0 Request processed in 121us (1341us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189056.709633:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6536 00000100:00100000:2.0:1776189056.709707:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000046fee2b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664279872:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189056.711007:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189056.712213:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664279808 00000100:00100000:3.0:1776189056.712284:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6537 00000100:00100000:3.0:1776189056.712290:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007df90c3e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664279808:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189056.712321:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189056.712389:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007df90c3e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664279808:12345-0@lo:13:osp-pre-0-0.0 Request processed in 98us (1387us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189056.712396:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6537 00000100:00100000:2.0:1776189056.712533:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004fbf075f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664279808:0@lo:13:osp-pre-0-0.0 00010000:02020000:1.0:1776189056.924823:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189057.027341:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189057.180294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189057.436290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189057.692956:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:3.0:1776189057.724525:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189057.724532:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189057.724535:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189057.724544:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9199 00000100:00080000:3.0:1776189057.724547:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776189057.724551:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189057.724570:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002ef57fef x1862459664280064/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189057.724609:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189057.724612:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189057.724614:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189057.724617:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9199 00000100:00080000:3.0:1776189057.724619:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776189057.724622:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189057.724631:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ea0541f6 x1862459664280192/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189057.724638:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189057.724640:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189057.724642:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189057.724644:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9199 00000100:00080000:3.0:1776189057.724646:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776189057.724649:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189057.724659:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e5080bed x1862459664280256/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189057.724751:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003afc110b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664279936:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189057.724843:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189057.724872:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000db2d4fd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664280384:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189057.724908:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189057.724932:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1776189057.724938:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004fbf075f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664280000:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189057.724961:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189057.724974:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000da51a899 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664280448:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189057.724992:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189057.725023:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664280000 00000100:00100000:1.0:1776189057.725060:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664280448 00000100:00100000:1.0:1776189057.725078:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9697 00000100:00100000:1.0:1776189057.725083:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c3185a52 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664280000:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189057.725129:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c3185a52 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664280000:12345-0@lo:400:kworker/u8:1.0 Request processed in 45us (167us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189057.725136:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9697 00000100:00100000:1.0:1776189057.725153:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9698 00000100:00100000:1.0:1776189057.725156:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e5117e34 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664280448:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189057.725221:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e5117e34 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664280448:12345-0@lo:400:kworker/u8:1.0 Request processed in 64us (228us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189057.725227:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9698 00000100:00100000:1.0:1776189057.725253:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664280384 00000100:00100000:1.0:1776189057.725272:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8353 00000100:00100000:1.0:1776189057.725276:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d97d7d85 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664280384:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189057.725302:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d97d7d85 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664280384:12345-0@lo:400:kworker/u8:1.0 Request processed in 25us (394us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189057.725308:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8353 00000100:00100000:1.0:1776189057.725336:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004fbf075f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664280000:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189057.725348:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000da51a899 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664280448:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189057.725355:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000db2d4fd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664280384:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189057.725533:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664279936 00000100:00100000:3.0:1776189057.725570:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1327 00000100:00100000:3.0:1776189057.725574:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b64e4d66 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664279936:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189057.725610:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b64e4d66 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664279936:12345-0@lo:400:kworker/u8:1.0 Request processed in 34us (768us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189057.725616:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1327 00000100:00100000:3.0:1776189057.725652:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003afc110b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664279936:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189057.725768:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002ef57fef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664280064:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189057.725798:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189057.725820:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ea0541f6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664280192:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189057.725841:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189057.725855:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e5080bed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664280256:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189057.725875:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189057.725905:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664280256 00000100:00100000:2.0:1776189057.725931:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9058 00000100:00100000:2.0:1776189057.725935:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003d6663c8 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664280256:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189057.725948:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189057.725981:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003d6663c8 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664280256:12345-0@lo:38:kworker/u8:1.0 Request processed in 45us (105us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189057.725986:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9058 00000100:00100000:2.0:1776189057.726003:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664280064 00000100:00100000:2.0:1776189057.726018:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664280192 00000100:00100000:2.0:1776189057.726033:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57176 00000100:00100000:2.0:1776189057.726036:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000046fee2b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664280064:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189057.726043:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189057.726064:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000046fee2b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664280064:12345-0@lo:38:kworker/u8:1.0 Request processed in 27us (266us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189057.726069:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57176 00000100:00100000:2.0:1776189057.726078:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57177 00000100:00100000:2.0:1776189057.726081:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c1af713e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664280192:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189057.726087:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189057.726102:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c1af713e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664280192:12345-0@lo:38:kworker/u8:1.0 Request processed in 20us (262us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189057.726107:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57177 00000100:00080000:2.0:1776189057.726133:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189057.726137:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189057.726140:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002ef57fef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664280064:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776189057.726147:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189057.726149:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189057.726152:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ea0541f6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664280192:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776189057.726158:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189057.726160:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189057.726162:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e5080bed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664280256:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189057.726218:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1d7bdea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664280128:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189057.726240:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189057.726253:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc904d7f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664280512:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189057.726270:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189057.726283:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1776189057.726288:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000101b4f14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664280320:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189057.726303:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189057.726312:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a0c7825a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664280576:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189057.726327:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189057.726346:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664280512 00000100:00100000:2.0:1776189057.726364:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664280320 00000100:00100000:2.0:1776189057.726374:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664280576 00000100:00100000:2.0:1776189057.726387:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8354 00000100:00100000:2.0:1776189057.726391:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000887d8a72 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664280512:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189057.726421:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000887d8a72 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664280512:12345-0@lo:400:kworker/u8:1.0 Request processed in 29us (151us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189057.726427:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8354 00000100:00100000:2.0:1776189057.726438:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8355 00000100:00100000:2.0:1776189057.726441:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000db8bee4c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664280320:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189057.726466:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000db8bee4c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664280320:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (162us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189057.726472:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8355 00000100:00100000:2.0:1776189057.726482:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8356 00000100:00100000:2.0:1776189057.726485:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c0205bdc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664280576:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189057.726606:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664280128 00000100:00100000:3.0:1776189057.726631:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9699 00000100:00100000:3.0:1776189057.726635:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000072b3348c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664280128:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189057.726681:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c0205bdc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664280576:12345-0@lo:400:kworker/u8:1.0 Request processed in 195us (353us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189057.726687:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8356 00000100:00100000:0.0:1776189057.726862:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000071a19d57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664280704:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189057.726890:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189057.726909:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776189057.726914:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f4820b3d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664280640:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189057.726933:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189057.726957:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664280704 00000100:00100000:0.0:1776189057.726981:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664280640 00000100:00100000:0.0:1776189057.726995:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6538 00000100:00100000:0.0:1776189057.726998:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e72d32a7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664280704:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776189057.727011:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189057.727038:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e72d32a7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664280704:12345-0@lo:13:osp-pre-0-1.0 Request processed in 39us (148us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189057.727043:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6538 00000100:00100000:0.0:1776189057.727053:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6539 00000100:00100000:0.0:1776189057.727056:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009b75a9f7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664280640:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189057.727063:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189057.727079:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009b75a9f7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664280640:12345-0@lo:13:osp-pre-1-1.0 Request processed in 23us (146us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189057.727083:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6539 00000100:00100000:0.0:1776189057.727114:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f4820b3d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664280640:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189057.727122:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000071a19d57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664280704:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189057.729527:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000072b3348c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664280128:12345-0@lo:400:kworker/u8:1.0 Request processed in 2882us (3278us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189057.729541:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9699 00000100:00100000:2.0:1776189057.729743:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000101b4f14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664280320:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189057.729762:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a0c7825a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664280576:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189057.729774:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1d7bdea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664280128:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189057.729785:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc904d7f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664280512:0@lo:400:kworker/u8:1.0 00010000:02020000:3.0:1776189057.949925:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189058.054378:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189058.204494:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189058.460402:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189058.716833:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189058.975339:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189059.068572:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189059.228982:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189059.484427:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189059.740844:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189059.997215:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189060.092439:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189060.252303:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189060.508648:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189060.769733:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189061.020569:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189061.117030:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189061.276290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189061.498824:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189061.498876:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189061.498965:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654940544 00000100:00100000:3.0:1776189061.499025:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9700 00000100:00100000:3.0:1776189061.499031:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d6ea5fef pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654940544:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1776189061.499108:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d6ea5fef pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654940544:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 76us (232us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189061.499123:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9700 00000100:00100000:0.0:1776189061.499685:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654940480 00000100:00100000:0.0:1776189061.499720:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1328 00000100:00100000:0.0:1776189061.499725:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f37c27d2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654940480:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776189061.500467:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f37c27d2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654940480:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 738us (1648us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189061.500476:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1328 00000100:00100000:1.0:1776189061.508212:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189061.508267:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189061.508389:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654940608 00000100:00100000:2.0:1776189061.508445:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57178 00000100:00100000:2.0:1776189061.508453:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006c8bdd15 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654940608:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:2.0:1776189061.508472:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189061.508534:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006c8bdd15 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654940608:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 82us (332us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189061.508542:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57178 00000100:00100000:3.0:1776189061.508623:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654940672 00000100:00100000:3.0:1776189061.508666:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8357 00000100:00100000:3.0:1776189061.508670:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004eaf09e3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2278:x1862459654940672:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1776189061.508709:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004eaf09e3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2278:x1862459654940672:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 38us (443us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189061.508715:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8357 00010000:02020000:3.0:1776189061.532599:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189061.789638:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189061.822471:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba2818a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664280768:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189061.822538:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189061.822758:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776189061.822767:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c89da359 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664280832:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189061.822792:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189061.822899:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664280768 00000100:00100000:2.0:1776189061.822945:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664280832 00000100:00100000:2.0:1776189061.822971:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6540 00000100:00100000:2.0:1776189061.822976:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000071a19d57 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664280768:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189061.823004:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189061.823047:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000071a19d57 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664280768:12345-0@lo:13:osp-pre-1-0.0 Request processed in 71us (510us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189061.823055:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6540 00000100:00100000:2.0:1776189061.823071:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6541 00000100:00100000:2.0:1776189061.823074:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f4820b3d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664280832:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189061.823090:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189061.823123:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f4820b3d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664280832:12345-0@lo:13:osp-pre-0-0.0 Request processed in 47us (331us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189061.823129:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6541 00000100:00100000:2.0:1776189061.823166:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c89da359 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664280832:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189061.823201:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba2818a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664280768:0@lo:13:osp-pre-1-0.0 00010000:02020000:1.0:1776189062.046258:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189062.140494:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189062.300280:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189062.556295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189062.815166:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189062.846054:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba2818a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664280960:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189062.846147:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189062.846224:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776189062.846231:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000098305fef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664280896:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189062.846253:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189062.847052:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664280960 00000100:00100000:0.0:1776189062.847116:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664280896 00000100:00100000:0.0:1776189062.847140:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6542 00000100:00100000:0.0:1776189062.847146:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a93ab5a8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664280960:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189062.847193:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189062.847247:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a93ab5a8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664280960:12345-0@lo:13:osp-pre-1-1.0 Request processed in 100us (1100us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189062.847255:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6542 00000100:00100000:0.0:1776189062.847271:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6543 00000100:00100000:0.0:1776189062.847275:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c0d0294 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664280896:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776189062.847287:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189062.847316:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c0d0294 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664280896:12345-0@lo:13:osp-pre-0-1.0 Request processed in 40us (1063us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189062.847323:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6543 00000100:00080000:3.0:1776189062.847406:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189062.847413:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189062.847416:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189062.847422:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9204 00000100:00080000:3.0:1776189062.847426:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00080000:3.0:1776189062.847430:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189062.847441:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000098305fef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664280896:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189062.847455:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000077c40bf x1862459664281152/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189062.847463:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba2818a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664280960:0@lo:13:osp-pre-1-1.0 00000100:00080000:3.0:1776189062.847488:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189062.847492:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1776189062.847493:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00080000:3.0:1776189062.847494:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189062.847500:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9204 00000100:00080000:3.0:1776189062.847502:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00100000:1.0:1776189062.847504:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d9c93ad2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664281216:0@lo:400:kworker/u8:0.0 00000100:00080000:3.0:1776189062.847507:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189062.847522:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d7babd95 x1862459664281280/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189062.847530:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189062.847533:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189062.847535:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189062.847538:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9204 00000100:00080000:3.0:1776189062.847539:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 6s 00000100:00080000:3.0:1776189062.847545:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189062.847547:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189062.847558:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000099b0e8b4 x1862459664281344/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189062.847589:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe5f85f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664281408:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189062.847611:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e463b423 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664281088:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189062.847620:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189062.847647:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189062.847668:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000229fc20a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664281536:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189062.847669:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664281216 00000100:00100000:3.0:1776189062.847696:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189062.847715:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1776189062.847720:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005c66c1fe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664281024:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189062.847748:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189062.847783:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c494ab9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664281472:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189062.847806:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189062.847838:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664281408 00000100:00100000:0.0:1776189062.847865:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664281024 00000100:00100000:3.0:1776189062.847867:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664281472 00000100:00100000:3.0:1776189062.847892:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8358 00000100:00100000:3.0:1776189062.847897:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004d99e80b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664281408:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189062.847905:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1329 00000100:00100000:0.0:1776189062.847910:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002e45a4e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664281024:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189062.847937:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004d99e80b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664281408:12345-0@lo:400:kworker/u8:0.0 Request processed in 39us (318us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189062.847944:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8358 00000100:00100000:0.0:1776189062.847960:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002e45a4e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664281024:12345-0@lo:400:kworker/u8:0.0 Request processed in 47us (211us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189062.847962:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8359 00000100:00100000:3.0:1776189062.847968:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009ac61361 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664281472:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189062.847973:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1329 00000100:00100000:3.0:1776189062.848006:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009ac61361 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664281472:12345-0@lo:400:kworker/u8:0.0 Request processed in 37us (199us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189062.848013:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8359 00000100:00100000:0.0:1776189062.848021:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005c66c1fe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664281024:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189062.848035:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c494ab9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664281472:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189062.848080:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000077c40bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664281152:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189062.848115:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe5f85f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664281408:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189062.848122:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189062.849576:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664281088 00000100:00100000:1.0:1776189062.849632:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664281536 00000100:00100000:1.0:1776189062.849654:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9701 00000100:00100000:1.0:1776189062.849659:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000000005173 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664281216:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189062.849729:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000000005173 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664281216:12345-0@lo:400:kworker/u8:0.0 Request processed in 70us (2187us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189062.849736:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9701 00000100:00100000:1.0:1776189062.849751:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9702 00000100:00100000:1.0:1776189062.849754:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003f45b220 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664281088:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189062.849786:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003f45b220 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664281088:12345-0@lo:400:kworker/u8:0.0 Request processed in 31us (2141us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189062.849792:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9702 00000100:00100000:1.0:1776189062.849802:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9703 00000100:00100000:1.0:1776189062.849805:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f600e06c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664281536:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189062.849831:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f600e06c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664281536:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (2135us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189062.849836:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9703 00000100:00100000:1.0:1776189062.849880:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d9c93ad2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664281216:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189062.849929:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e463b423 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664281088:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189062.849939:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000229fc20a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664281536:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189062.857456:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d7babd95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664281280:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189062.857524:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189062.857550:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000099b0e8b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664281344:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189062.857576:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189062.857633:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664281344 00000100:00100000:3.0:1776189062.857668:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9059 00000100:00100000:3.0:1776189062.857673:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000e75ea3b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664281344:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189062.857689:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189062.857726:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000e75ea3b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664281344:12345-0@lo:38:kworker/u8:0.0 Request processed in 51us (149us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189062.857833:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9059 00000100:00080000:3.0:1776189062.857878:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189062.857882:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189062.857886:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000099b0e8b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664281344:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189062.858493:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664281152 00000100:00100000:1.0:1776189062.858536:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664281280 00000100:00100000:1.0:1776189062.858556:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57179 00000100:00100000:1.0:1776189062.858560:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a17ea634 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664281152:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189062.858573:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189062.858620:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a17ea634 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664281152:12345-0@lo:38:kworker/u8:0.0 Request processed in 59us (10502us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189062.858627:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57179 00000100:00100000:1.0:1776189062.858640:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57180 00000100:00100000:1.0:1776189062.858643:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f5d3008d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664281280:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189062.858650:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189062.858672:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f5d3008d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664281280:12345-0@lo:38:kworker/u8:0.0 Request processed in 27us (1149us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189062.858677:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57180 00000100:00080000:3.0:1776189062.863303:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189062.863313:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189062.863321:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000077c40bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664281152:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776189062.863334:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189062.863337:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189062.863340:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d7babd95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664281280:0@lo:38:kworker/u8:0.0 00010000:02020000:1.0:1776189063.068696:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189063.166626:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189063.324525:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189063.585115:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189063.837928:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189064.095689:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189064.188462:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189064.348321:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189064.604424:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189064.860306:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189065.118282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189065.212377:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189065.372635:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189065.628288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189065.887220:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189066.144940:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189066.236554:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189066.396246:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189066.618435:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189066.618479:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189066.618516:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189066.618600:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654940736 00000100:00100000:2.0:1776189066.618657:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1330 00000100:00100000:2.0:1776189066.618664:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002ea0417b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654940736:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776189066.618680:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654940864 00000100:00100000:0.0:1776189066.618723:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654940800 00000100:00100000:2.0:1776189066.618732:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002ea0417b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654940736:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 67us (252us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189066.618741:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1330 00000100:00100000:0.0:1776189066.618754:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9704 00000100:00100000:0.0:1776189066.618759:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ab3412dc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654940800:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776189066.618798:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ab3412dc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654940800:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 37us (281us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189066.618805:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57181 00000100:00100000:0.0:1776189066.618816:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9704 00000100:00100000:2.0:1776189066.618817:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000021503dd8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654940864:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:2.0:1776189066.618835:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189066.618865:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000021503dd8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654940864:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 53us (437us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189066.618870:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57181 00000100:00100000:0.0:1776189066.626450:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189066.627654:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654940928 00000100:00100000:2.0:1776189066.627719:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8360 00000100:00100000:2.0:1776189066.627725:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000611c527e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2279:x1862459654940928:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1776189066.627814:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000611c527e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2279:x1862459654940928:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 88us (1373us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189066.627822:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8360 00010000:02020000:3.0:1776189066.652274:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189066.908675:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189066.940506:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776189066.940528:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b29992fd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664281600:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189066.940589:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776189066.940590:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189066.940603:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d7babd95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664281664:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189066.940649:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189066.940727:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664281600 00000100:00100000:3.0:1776189066.940775:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664281664 00000100:00100000:3.0:1776189066.940800:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6544 00000100:00100000:3.0:1776189066.940805:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003afc110b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664281600:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776189066.940833:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189066.940852:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6545 00000100:00100000:1.0:1776189066.940858:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c5a99ce9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664281664:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189066.940877:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189066.940877:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003afc110b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664281600:12345-0@lo:13:osp-pre-1-0.0 Request processed in 69us (287us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189066.940888:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6544 00000100:00100000:1.0:1776189066.940916:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c5a99ce9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664281664:12345-0@lo:13:osp-pre-0-0.0 Request processed in 57us (266us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189066.940923:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6545 00000100:00100000:1.0:1776189066.940960:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b29992fd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664281600:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189066.941224:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d7babd95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664281664:0@lo:13:osp-pre-0-0.0 00010000:02020000:3.0:1776189067.164285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189067.262100:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189067.420289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189067.678725:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189067.932377:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:3.0:1776189067.964875:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189067.964882:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189067.964885:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189067.964893:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9209 00000100:00080000:3.0:1776189067.964897:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00080000:3.0:1776189067.964901:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189067.964920:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000078cf1bd6 x1862459664281856/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189067.964959:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189067.964962:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1776189067.965115:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000077c40bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664281728:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189067.965187:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000099b0e8b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664281792:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189067.965298:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000078cf1bd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664281856:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776189067.966160:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189067.966167:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9209 00000100:00080000:3.0:1776189067.966170:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:3.0:1776189067.966188:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189067.966212:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000229fc20a x1862459664282112/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189067.966224:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189067.966228:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189067.966229:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189067.966232:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9209 00000100:00080000:3.0:1776189067.966234:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 11s 00000100:00080000:3.0:1776189067.966237:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189067.966247:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e463b423 x1862459664282176/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189067.966317:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d7babd95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664281920:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189067.974440:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189067.974506:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d9c93ad2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664282240:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189067.974552:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189067.974572:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c89da359 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664282496:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189067.974592:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189067.974611:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1776189067.974617:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000673b41e8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664282048:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189067.974651:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189067.974667:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005c66c1fe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664282432:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189067.974686:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189067.974721:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e3a29c53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664281984:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189067.974740:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189067.974766:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c494ab9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664282368:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189067.974785:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189067.974814:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664281792 00000100:00100000:1.0:1776189067.974847:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664282368 00000100:00100000:1.0:1776189067.974861:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9705 00000100:00100000:1.0:1776189067.974866:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004fbf075f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664281792:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189067.974907:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004fbf075f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664281792:12345-0@lo:400:kworker/u8:1.0 Request processed in 41us (477us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189067.974914:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9705 00000100:00100000:1.0:1776189067.974930:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9706 00000100:00100000:1.0:1776189067.974932:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000239d0820 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664282368:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189067.975202:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000239d0820 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664282368:12345-0@lo:400:kworker/u8:1.0 Request processed in 268us (416us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189067.975209:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9706 00000100:00100000:1.0:1776189067.975230:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664282240 00000100:00100000:1.0:1776189067.975251:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664282496 00000100:00100000:1.0:1776189067.975267:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664282432 00000100:00100000:1.0:1776189067.975289:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8361 00000100:00100000:1.0:1776189067.975293:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000088c2a62e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664282240:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189067.975320:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000088c2a62e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664282240:12345-0@lo:400:kworker/u8:1.0 Request processed in 27us (769us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189067.975326:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8361 00000100:00100000:1.0:1776189067.975335:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8362 00000100:00100000:1.0:1776189067.975338:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000540d168e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664282496:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189067.975359:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000540d168e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664282496:12345-0@lo:400:kworker/u8:1.0 Request processed in 20us (766us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189067.975364:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8362 00000100:00100000:1.0:1776189067.975373:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8363 00000100:00100000:1.0:1776189067.975376:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005d3cfd23 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664282432:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189067.975398:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005d3cfd23 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664282432:12345-0@lo:400:kworker/u8:1.0 Request processed in 20us (711us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189067.975403:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8363 00000100:00100000:1.0:1776189067.975429:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664282048 00000100:00100000:1.0:1776189067.975449:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664281984 00000100:00100000:1.0:1776189067.975463:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6546 00000100:00100000:1.0:1776189067.975466:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a109d002 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664282048:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189067.975480:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189067.975509:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a109d002 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664282048:12345-0@lo:13:osp-pre-1-1.0 Request processed in 42us (858us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189067.975514:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6546 00000100:00100000:1.0:1776189067.975525:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6547 00000100:00100000:1.0:1776189067.975528:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009fe6d891 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664281984:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189067.975536:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189067.975558:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009fe6d891 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664281984:12345-0@lo:13:osp-pre-0-1.0 Request processed in 29us (818us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189067.975563:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6547 00000100:00100000:1.0:1776189067.975611:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e3a29c53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664281984:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189067.975621:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c494ab9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664282368:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189067.975674:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000673b41e8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664282048:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189067.975683:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005c66c1fe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664282432:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189067.975690:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d9c93ad2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664282240:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189067.975697:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c89da359 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664282496:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189067.975705:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000099b0e8b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664281792:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189067.976038:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189067.978890:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664281728 00000100:00100000:1.0:1776189067.978964:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1331 00000100:00100000:1.0:1776189067.978970:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a9501f9f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664281728:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189067.979064:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a9501f9f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664281728:12345-0@lo:400:kworker/u8:1.0 Request processed in 93us (3028us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189067.979071:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1331 00000100:00100000:0.0:1776189067.979492:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000077c40bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664281728:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189067.981862:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189067.981915:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000229fc20a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664282112:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189067.981947:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189067.981991:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e463b423 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664282176:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189067.982016:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189067.982075:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664282176 00000100:00100000:2.0:1776189067.982120:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9060 00000100:00100000:2.0:1776189067.982125:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000040b53b34 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664282176:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189067.982146:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189067.982252:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664281856 00000100:00100000:1.0:1776189067.982277:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664282112 00000100:00100000:1.0:1776189067.982291:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57182 00000100:00100000:1.0:1776189067.982294:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000061dd8d04 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664281856:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189067.982303:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189067.982340:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000061dd8d04 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664281856:12345-0@lo:38:kworker/u8:1.0 Request processed in 44us (484us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189067.982347:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57182 00000100:00100000:1.0:1776189067.982363:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57183 00000100:00100000:1.0:1776189067.982366:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000205cde5a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664282112:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189067.982373:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189067.982396:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000205cde5a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664282112:12345-0@lo:38:kworker/u8:1.0 Request processed in 29us (449us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189067.982401:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57183 00000100:00100000:3.0:1776189067.983093:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189067.983231:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe5f85f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664282304:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189067.983261:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189067.983365:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664282304 00000100:00100000:1.0:1776189067.983398:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8364 00000100:00100000:1.0:1776189067.983403:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b1d7bdea pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664282304:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189067.983466:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b1d7bdea pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664282304:12345-0@lo:400:kworker/u8:1.0 Request processed in 62us (205us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189067.983473:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8364 00000100:00100000:2.0:1776189067.983492:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000040b53b34 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664282176:12345-0@lo:38:kworker/u8:1.0 Request processed in 1363us (1473us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189067.983502:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9060 00000100:00080000:2.0:1776189067.983542:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189067.983546:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189067.983550:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000229fc20a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664282112:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776189067.983560:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189067.983562:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189067.983565:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e463b423 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664282176:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776189067.983574:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189067.983576:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189067.983579:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000078cf1bd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664281856:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189067.983947:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe5f85f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664282304:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189067.983956:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664281920 00000100:00100000:0.0:1776189067.985596:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9707 00000100:00100000:0.0:1776189067.985693:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cc904d7f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664281920:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189067.985788:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cc904d7f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664281920:12345-0@lo:400:kworker/u8:1.0 Request processed in 97us (2698us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189067.985863:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d7babd95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664281920:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189067.990430:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9707 00010000:02020000:3.0:1776189068.189015:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189068.284837:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189068.444317:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189068.700265:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189068.956247:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189069.212793:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189069.308445:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189069.468834:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189069.724559:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189069.980505:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189070.236905:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189070.332379:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189070.493007:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189070.748329:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189071.004511:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189071.260305:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189071.356382:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189071.516293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189071.739289:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189071.739345:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189071.739391:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189071.739426:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189071.739536:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654940992 00000100:00100000:2.0:1776189071.739594:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1332 00000100:00100000:2.0:1776189071.739601:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004c8f754f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654940992:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189071.739669:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004c8f754f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654940992:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 67us (325us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189071.739677:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1332 00000100:00100000:1.0:1776189071.740134:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654941120 00000100:00100000:1.0:1776189071.740170:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654941056 00000100:00100000:1.0:1776189071.740222:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9708 00000100:00100000:1.0:1776189071.740226:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e3a29c53 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654941056:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189071.740264:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e3a29c53 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654941056:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 37us (838us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189071.740271:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9708 00000100:00100000:1.0:1776189071.740284:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57184 00000100:00100000:1.0:1776189071.740288:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a6fd34e4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654941120:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776189071.740302:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189071.740327:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a6fd34e4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654941120:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 38us (1046us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189071.740332:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57184 00000100:00100000:3.0:1776189071.740993:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654941184 00000100:00100000:3.0:1776189071.741046:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8365 00000100:00100000:3.0:1776189071.741050:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001e4997ad pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2279:x1862459654941184:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189071.741102:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001e4997ad pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2279:x1862459654941184:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 50us (1710us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189071.741108:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8365 00010000:02020000:3.0:1776189071.776088:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189072.028278:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189072.060594:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d7babd95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664282560:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189072.060661:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189072.060750:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664282560 00000100:00100000:3.0:1776189072.060800:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6548 00000100:00100000:3.0:1776189072.060805:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000098305fef pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664282560:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189072.060832:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189072.060881:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000098305fef pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664282560:12345-0@lo:13:osp-pre-0-0.0 Request processed in 75us (220us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189072.060896:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6548 00000100:00100000:0.0:1776189072.061025:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d7babd95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664282560:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189072.061538:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776189072.061549:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000914d0a5f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664282624:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189072.061577:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189072.061644:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664282624 00000100:00100000:2.0:1776189072.061677:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6549 00000100:00100000:2.0:1776189072.061682:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f12977ad pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664282624:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189072.061700:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189072.061737:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f12977ad pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664282624:12345-0@lo:13:osp-pre-1-0.0 Request processed in 55us (160us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189072.061743:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6549 00000100:00100000:0.0:1776189072.061822:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000914d0a5f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664282624:0@lo:13:osp-pre-1-0.0 00010000:02020000:3.0:1776189072.284718:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189072.380457:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189072.540290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189072.796341:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189073.052291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:3.0:1776189073.084348:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189073.084358:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189073.084362:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189073.084372:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9214 00000100:00080000:3.0:1776189073.084376:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00100000:1.0:1776189073.084380:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a0c7825a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664282752:0@lo:400:kworker/u8:0.0 00000100:00080000:3.0:1776189073.084384:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189073.084407:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000101b4f14 x1862459664282816/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189073.084441:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189073.084445:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189073.084446:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189073.084451:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9214 00000100:00100000:1.0:1776189073.084455:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1776189073.084456:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1776189073.084462:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189073.084476:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ea0541f6 x1862459664282944/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189073.084485:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189073.084487:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1776189073.084502:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776189073.084509:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe5f85f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664282688:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189073.084539:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189073.084609:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e5080bed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664282880:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189073.084645:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189073.084649:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664282688 00000100:00100000:0.0:1776189073.084678:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776189073.084688:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004dd53b72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664283008:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189073.084688:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1333 00000100:00100000:1.0:1776189073.084698:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a9f52f65 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664282688:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189073.084719:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189073.084752:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664282752 00000100:00100000:1.0:1776189073.084756:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a9f52f65 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664282688:12345-0@lo:400:kworker/u8:0.0 Request processed in 58us (215us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189073.084765:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1333 00000100:00100000:0.0:1776189073.084783:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664282880 00000100:00100000:0.0:1776189073.084806:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9709 00000100:00100000:0.0:1776189073.084811:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000932d0506 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664282752:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189073.084817:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe5f85f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664282688:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189073.084853:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000932d0506 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664282752:12345-0@lo:400:kworker/u8:0.0 Request processed in 40us (401us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189073.084859:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9709 00000100:00080000:3.0:1776189073.084947:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1776189073.084951:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a0c7825a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664282752:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189073.088318:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9710 00000100:00100000:0.0:1776189073.088332:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ff579061 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664282880:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189073.088421:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ff579061 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664282880:12345-0@lo:400:kworker/u8:0.0 Request processed in 91us (3777us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189073.092144:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664283008 00000100:00100000:1.0:1776189073.092209:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6550 00000100:00100000:1.0:1776189073.092226:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002ea466a7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664283008:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189073.092250:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189073.092301:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000acf34144 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664283072:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189073.092322:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002ea466a7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664283008:12345-0@lo:13:osp-pre-0-1.0 Request processed in 106us (7603us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189073.092330:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6550 00000100:00100000:2.0:1776189073.092352:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189073.093339:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000101b4f14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664282816:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189073.093387:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189073.093410:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664282816 00000100:00100000:2.0:1776189073.093436:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57185 00000100:00100000:2.0:1776189073.093440:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fdda22fa pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664282816:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189073.093455:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189073.093482:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fdda22fa pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664282816:12345-0@lo:38:kworker/u8:0.0 Request processed in 41us (96us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189073.093488:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57185 00000100:00100000:2.0:1776189073.093503:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ea0541f6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664282944:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189073.093523:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189073.093540:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664282944 00000100:00100000:2.0:1776189073.093555:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57186 00000100:00100000:2.0:1776189073.093557:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002c842ea9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664282944:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189073.093563:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189073.093581:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002c842ea9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664282944:12345-0@lo:38:kworker/u8:0.0 Request processed in 23us (59us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189073.093585:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57186 00000100:00080000:2.0:1776189073.093608:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189073.093612:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189073.093615:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000101b4f14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664282816:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776189073.093621:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189073.093623:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189073.093626:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ea0541f6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664282944:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189073.093710:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004dd53b72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664283008:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189073.093721:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e5080bed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664282880:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189073.098484:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9710 00000100:00080000:3.0:1776189073.098848:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9214 00000100:00080000:3.0:1776189073.098853:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1776189073.098857:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189073.098916:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000db2d4fd6 x1862459664283136/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189073.099026:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ea0541f6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664283328:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189073.099037:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664283072 00000100:00100000:2.0:1776189073.099084:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6551 00000100:00100000:3.0:1776189073.099089:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189073.099092:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003fb63a1c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664283072:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189073.099114:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189073.099118:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776189073.099125:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e5080bed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664283264:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189073.099168:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003fb63a1c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664283072:12345-0@lo:13:osp-pre-1-1.0 Request processed in 74us (6816us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189073.099170:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189073.099194:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6551 00000100:00100000:3.0:1776189073.099224:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000da51a899 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664283200:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189073.099248:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189073.099276:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664283264 00000100:00100000:3.0:1776189073.099301:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664283200 00000100:00100000:3.0:1776189073.099315:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8366 00000100:00100000:3.0:1776189073.099320:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0c7825a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664283264:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189073.099354:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0c7825a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664283264:12345-0@lo:400:kworker/u8:0.0 Request processed in 33us (189us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189073.099360:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8366 00000100:00100000:3.0:1776189073.099372:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8367 00000100:00100000:3.0:1776189073.099376:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007fe5f85f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664283200:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189073.099405:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007fe5f85f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664283200:12345-0@lo:400:kworker/u8:0.0 Request processed in 28us (156us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189073.099411:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8367 00000100:00100000:3.0:1776189073.099442:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e5080bed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664283264:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189073.099468:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000da51a899 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664283200:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189073.099497:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000acf34144 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664283072:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189073.099697:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664283328 00000100:00100000:0.0:1776189073.099734:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9711 00000100:00100000:0.0:1776189073.099738:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000101b4f14 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664283328:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189073.099775:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000101b4f14 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664283328:12345-0@lo:400:kworker/u8:0.0 Request processed in 36us (688us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189073.099781:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9711 00000100:00100000:0.0:1776189073.099799:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000db2d4fd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664283136:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189073.099825:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189073.099848:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664283136 00000100:00100000:0.0:1776189073.099878:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9061 00000100:00100000:0.0:1776189073.099881:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000039f4c6c3 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664283136:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189073.099894:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189073.099919:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000039f4c6c3 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664283136:12345-0@lo:38:kworker/u8:0.0 Request processed in 37us (94us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189073.099927:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9061 00000100:00080000:0.0:1776189073.099953:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189073.099956:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189073.099959:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000db2d4fd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664283136:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189073.103156:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ea0541f6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664283328:0@lo:400:kworker/u8:0.0 00010000:02020000:3.0:1776189073.308344:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189073.404569:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189073.564545:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189073.820602:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189074.078857:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189074.332334:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189074.428915:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189074.588298:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189074.844309:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189075.100302:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189075.356298:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189075.452459:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189075.612449:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189075.868615:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189076.125377:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189076.380808:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189076.476397:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189076.638735:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189076.859517:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189076.859685:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654941376 00000100:00100000:0.0:1776189076.859737:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57187 00000100:00100000:0.0:1776189076.859744:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ea0541f6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654941376:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776189076.859765:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189076.859845:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ea0541f6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654941376:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 100us (334us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189076.859851:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57187 00000100:00100000:0.0:1776189076.861443:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189076.861490:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189076.861532:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189076.861626:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654941248 00000100:00100000:2.0:1776189076.861678:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1334 00000100:00100000:2.0:1776189076.861683:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000be36d03e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654941248:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189076.861745:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000be36d03e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654941248:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 60us (310us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189076.861751:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1334 00000100:00100000:3.0:1776189076.861916:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654941312 00000100:00100000:3.0:1776189076.861954:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9712 00000100:00100000:3.0:1776189076.861958:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000023a5b7b3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654941312:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189076.861993:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000023a5b7b3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654941312:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 34us (460us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189076.862003:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9712 00000100:00100000:1.0:1776189076.862838:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654941440 00000100:00100000:1.0:1776189076.862881:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8368 00000100:00100000:1.0:1776189076.862886:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fc3b4133 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654941440:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189076.862933:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fc3b4133 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654941440:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 46us (1443us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189076.862939:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8368 00010000:02020000:1.0:1776189076.892394:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189077.148408:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189077.182202:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000da51a899 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664283392:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189077.182285:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189077.182395:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664283392 00000100:00100000:3.0:1776189077.182446:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6552 00000100:00100000:3.0:1776189077.182452:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004ac5ba05 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664283392:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189077.182481:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189077.182532:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004ac5ba05 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664283392:12345-0@lo:13:osp-pre-0-0.0 Request processed in 80us (248us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189077.182539:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6552 00000100:00100000:0.0:1776189077.182800:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000da51a899 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664283392:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189077.183242:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094ac8433 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664283456:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189077.183285:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189077.183406:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664283456 00000100:00100000:0.0:1776189077.183443:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6553 00000100:00100000:0.0:1776189077.183448:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a03700f8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664283456:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189077.183467:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189077.183916:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a03700f8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664283456:12345-0@lo:13:osp-pre-1-0.0 Request processed in 465us (629us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189077.183924:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6553 00000100:00100000:1.0:1776189077.183981:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094ac8433 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664283456:0@lo:13:osp-pre-1-0.0 00010000:02020000:1.0:1776189077.406065:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189077.501531:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189077.660288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189077.916584:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189078.174123:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189078.204544:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e139d197 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664283520:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189078.204625:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189078.204786:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664283520 00000100:00100000:2.0:1776189078.204836:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6554 00000100:00100000:2.0:1776189078.204842:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002ae5cc97 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664283520:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189078.204866:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189078.204909:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002ae5cc97 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664283520:12345-0@lo:13:osp-pre-0-1.0 Request processed in 65us (283us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189078.204916:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6554 00000100:00100000:0.0:1776189078.205220:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e139d197 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664283520:0@lo:13:osp-pre-0-1.0 00000100:00080000:3.0:1776189078.205551:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189078.205557:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189078.205560:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189078.205564:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9220 00000100:00080000:3.0:1776189078.205567:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189078.205571:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189078.205587:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000078cf1bd6 x1862459664283712/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189078.205607:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189078.205610:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189078.205611:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189078.205614:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9220 00000100:00080000:3.0:1776189078.205616:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189078.205619:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189078.205627:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000229fc20a x1862459664283840/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189078.205635:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189078.205638:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189078.205639:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189078.205641:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9220 00000100:00080000:3.0:1776189078.205643:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189078.205646:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189078.205654:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000077c40bf x1862459664283904/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189078.205691:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e5080bed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664283584:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189078.205722:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189078.205735:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c89da359 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664284032:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189078.205880:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000078cf1bd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664283712:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189078.205907:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189078.205924:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000229fc20a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664283840:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189078.205944:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189078.205958:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000077c40bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664283904:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189078.205978:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189078.206014:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664283584 00000100:00100000:0.0:1776189078.206045:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1335 00000100:00100000:0.0:1776189078.206049:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000816fca97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664283584:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189078.206083:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000816fca97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664283584:12345-0@lo:400:kworker/u8:0.0 Request processed in 33us (360us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189078.206089:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1335 00000100:00100000:0.0:1776189078.206133:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664283712 00000100:00100000:3.0:1776189078.206138:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189078.206152:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664283840 00000100:00100000:0.0:1776189078.206187:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57188 00000100:00100000:0.0:1776189078.206192:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000005ed8ead pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664283712:12345-0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189078.206203:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e5080bed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664283584:0@lo:400:kworker/u8:0.0 00010000:02000000:0.0:1776189078.206205:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189078.206222:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776189078.206236:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000099b0e8b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664283968:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189078.206241:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000005ed8ead pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664283712:12345-0@lo:38:kworker/u8:0.0 Request processed in 47us (333us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189078.206247:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57188 00000100:00100000:0.0:1776189078.206259:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57189 00000100:00100000:0.0:1776189078.206262:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005ee0b9e5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664283840:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189078.206269:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189078.206276:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189078.206301:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005ee0b9e5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664283840:12345-0@lo:38:kworker/u8:0.0 Request processed in 36us (356us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189078.206308:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57189 00000100:00100000:3.0:1776189078.206311:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e463b423 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664283776:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189078.206326:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664283904 00000100:00100000:0.0:1776189078.206346:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9062 00000100:00100000:0.0:1776189078.206350:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003189c953 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664283904:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189078.206356:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189078.206378:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003189c953 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664283904:12345-0@lo:38:kworker/u8:0.0 Request processed in 27us (400us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189078.206383:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9062 00000100:00080000:0.0:1776189078.206411:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189078.206415:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189078.206418:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000078cf1bd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664283712:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776189078.206426:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189078.206428:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189078.206431:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000229fc20a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664283840:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776189078.206437:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189078.206439:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189078.206441:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000077c40bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664283904:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189078.207890:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189078.207928:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005c66c1fe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664284160:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189078.207962:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189078.207983:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1776189078.207989:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d7babd95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664283648:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189078.208010:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189078.208027:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d9c93ad2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664284096:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189078.208047:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189078.208083:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664283776 00000100:00100000:3.0:1776189078.208116:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664283648 00000100:00100000:3.0:1776189078.208126:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664284096 00000100:00100000:3.0:1776189078.208136:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9713 00000100:00100000:3.0:1776189078.208140:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000077c40bf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664283776:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189078.208214:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000077c40bf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664283776:12345-0@lo:400:kworker/u8:0.0 Request processed in 72us (330us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189078.208222:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9713 00000100:00100000:3.0:1776189078.208248:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9714 00000100:00100000:3.0:1776189078.208252:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000078cf1bd6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664283648:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189078.208284:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000078cf1bd6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664283648:12345-0@lo:400:kworker/u8:0.0 Request processed in 31us (273us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189078.208289:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9714 00000100:00100000:3.0:1776189078.208299:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9715 00000100:00100000:3.0:1776189078.208302:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000da51a899 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664284096:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189078.208324:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000da51a899 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664284096:12345-0@lo:400:kworker/u8:0.0 Request processed in 21us (277us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189078.208329:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9715 00000100:00100000:3.0:1776189078.211023:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664284032 00000100:00100000:3.0:1776189078.211073:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664283968 00000100:00100000:3.0:1776189078.211096:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664284160 00000100:00100000:3.0:1776189078.211118:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8369 00000100:00100000:3.0:1776189078.211124:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000c494ab9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664284032:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189078.211227:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000c494ab9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664284032:12345-0@lo:400:kworker/u8:0.0 Request processed in 102us (5092us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189078.211951:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8369 00000100:00100000:3.0:1776189078.211973:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8370 00000100:00100000:3.0:1776189078.211978:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e5080bed pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664283968:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189078.212022:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e5080bed pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664283968:12345-0@lo:400:kworker/u8:0.0 Request processed in 44us (5747us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189078.212028:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8370 00000100:00100000:3.0:1776189078.212039:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8371 00000100:00100000:3.0:1776189078.212044:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000229fc20a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664284160:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189078.212071:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000229fc20a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664284160:12345-0@lo:400:kworker/u8:0.0 Request processed in 27us (4109us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189078.212077:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8371 00000100:00100000:2.0:1776189078.214908:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d7babd95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664283648:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189078.214930:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d9c93ad2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664284096:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189078.217966:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e463b423 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664283776:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189078.218011:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005c66c1fe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664284160:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189078.218331:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000099b0e8b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664283968:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189078.218344:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c89da359 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664284032:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189078.218362:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776189078.218372:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003fb63a1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664284224:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189078.218433:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189078.218549:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664284224 00000100:00100000:1.0:1776189078.218612:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6555 00000100:00100000:1.0:1776189078.218619:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e139d197 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664284224:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189078.218645:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189078.218695:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e139d197 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664284224:12345-0@lo:13:osp-pre-1-1.0 Request processed in 77us (263us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189078.218703:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6555 00000100:00100000:2.0:1776189078.219647:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003fb63a1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664284224:0@lo:13:osp-pre-1-1.0 00010000:02020000:3.0:1776189078.428288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189078.525836:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189078.684321:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189078.940304:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189079.196928:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189079.452283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189079.549233:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189079.708281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189079.965688:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189080.220339:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189080.476294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189080.572406:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189080.732261:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189080.988280:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189081.244290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189081.500287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189081.596846:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189081.757759:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189081.978593:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189081.978639:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189081.978679:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189081.978815:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654941696 00000100:00100000:3.0:1776189081.978836:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654941568 00000100:00100000:3.0:1776189081.978881:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9716 00000100:00100000:3.0:1776189081.978888:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001a975e34 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654941568:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1776189081.978927:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8372 00000100:00100000:1.0:1776189081.978934:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c8b239a6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654941696:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1776189081.978958:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001a975e34 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654941568:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 69us (373us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189081.978966:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9716 00000100:00100000:1.0:1776189081.978976:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c8b239a6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2281:x1862459654941696:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 40us (305us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189081.978983:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8372 00000100:00100000:2.0:1776189081.979051:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654941504 00000100:00100000:2.0:1776189081.979083:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1336 00000100:00100000:2.0:1776189081.979089:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c4f71e57 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654941504:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1776189081.979124:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c4f71e57 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654941504:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 34us (484us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189081.979146:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1336 00000100:00100000:3.0:1776189081.981068:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189081.981343:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654941632 00000100:00100000:2.0:1776189081.981667:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57190 00000100:00100000:2.0:1776189081.981683:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000db2d4fd6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654941632:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:2.0:1776189081.981702:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189081.981757:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000db2d4fd6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654941632:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 73us (696us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189081.981763:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57190 00010000:02020000:3.0:1776189082.015316:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189082.268366:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189082.300451:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003fb63a1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664284288:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189082.300521:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189082.300728:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664284288 00000100:00100000:1.0:1776189082.300776:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6556 00000100:00100000:1.0:1776189082.300782:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c89da359 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664284288:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189082.300812:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189082.300812:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000099b0e8b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664284352:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189082.300850:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189082.300869:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c89da359 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664284288:12345-0@lo:13:osp-pre-1-0.0 Request processed in 85us (347us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189082.300877:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6556 00000100:00100000:2.0:1776189082.300894:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664284352 00000100:00100000:2.0:1776189082.300920:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6557 00000100:00100000:2.0:1776189082.300925:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000094ac8433 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664284352:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189082.300940:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189082.300978:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000094ac8433 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664284352:12345-0@lo:13:osp-pre-0-0.0 Request processed in 53us (129us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189082.300983:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6557 00000100:00100000:1.0:1776189082.301341:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000099b0e8b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664284352:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189082.301796:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003fb63a1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664284288:0@lo:13:osp-pre-1-0.0 00010000:02020000:3.0:1776189082.524247:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189082.622473:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189082.780301:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189083.036431:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189083.293424:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:3.0:1776189083.324353:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189083.324360:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189083.324364:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189083.324373:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9225 00000100:00080000:3.0:1776189083.324376:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189083.324382:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189083.324402:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d9c93ad2 x1862459664284544/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189083.324427:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189083.324430:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189083.324432:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189083.324436:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9225 00000100:00080000:3.0:1776189083.324438:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189083.324441:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189083.324451:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000099b0e8b4 x1862459664284672/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189083.324460:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1776189083.324461:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00080000:3.0:1776189083.324466:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189083.324468:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1776189083.324482:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005c66c1fe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664284416:0@lo:400:kworker/u8:1.0 00000100:00080000:3.0:1776189083.324483:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9225 00000100:00080000:3.0:1776189083.324487:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189083.324491:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189083.324508:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b7a82fb0 x1862459664284736/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189083.324555:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189083.324583:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d7babd95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664284608:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189083.324588:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d92ffa01 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664284800:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189083.324625:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189083.324630:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189083.324662:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000053ba7e65 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664284992:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189083.324666:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000022f2aa27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664285056:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189083.324696:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189083.324713:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664284416 00000100:00100000:3.0:1776189083.324721:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1776189083.324728:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e463b423 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664284480:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189083.324754:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189083.324757:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1337 00000100:00100000:1.0:1776189083.324763:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b1a6ecdc pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664284416:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189083.324806:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b1a6ecdc pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664284416:12345-0@lo:400:kworker/u8:1.0 Request processed in 42us (254us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189083.324814:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1337 00000100:00100000:1.0:1776189083.324857:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664284800 00000100:00100000:1.0:1776189083.324881:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664284992 00000100:00100000:1.0:1776189083.324898:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8373 00000100:00100000:1.0:1776189083.324902:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cf6b5d65 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664284800:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189083.324923:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004ae527b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664284928:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189083.324937:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cf6b5d65 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664284800:12345-0@lo:400:kworker/u8:1.0 Request processed in 33us (310us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189083.324944:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8373 00000100:00100000:3.0:1776189083.324954:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189083.324957:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8374 00000100:00100000:1.0:1776189083.324963:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000684e24e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664284992:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189083.324998:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000684e24e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664284992:12345-0@lo:400:kworker/u8:1.0 Request processed in 35us (304us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189083.325004:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8374 00000100:00100000:1.0:1776189083.325027:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189083.325051:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005c66c1fe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664284416:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189083.325092:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d92ffa01 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664284800:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189083.325101:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776189083.325107:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f059006a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664284864:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189083.325131:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189083.325160:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664285056 00000100:00100000:0.0:1776189083.325163:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d9c93ad2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664284544:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189083.325209:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189083.325224:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000099b0e8b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664284672:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189083.325243:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189083.325258:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b7a82fb0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664284736:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189083.325277:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189083.325305:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664284864 00000100:00100000:0.0:1776189083.325329:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8375 00000100:00100000:0.0:1776189083.325333:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000decf8864 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664284864:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189083.325359:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8376 00000100:00100000:1.0:1776189083.325364:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b34cb071 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664285056:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189083.325374:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000decf8864 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664284864:12345-0@lo:400:kworker/u8:1.0 Request processed in 38us (242us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189083.325381:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8375 00000100:00100000:1.0:1776189083.325401:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b34cb071 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664285056:12345-0@lo:400:kworker/u8:1.0 Request processed in 35us (373us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189083.325408:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8376 00000100:00100000:1.0:1776189083.325445:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000053ba7e65 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664284992:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189083.325466:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664284480 00000100:00100000:1.0:1776189083.325486:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664284736 00000100:00100000:0.0:1776189083.325505:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664284928 00000100:00100000:1.0:1776189083.325513:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9063 00000100:00100000:1.0:1776189083.325519:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004ac5ba05 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664284736:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189083.325522:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664284544 00000100:00100000:0.0:1776189083.325533:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664284672 00010000:02000000:1.0:1776189083.325534:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189083.325545:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9717 00000100:00100000:0.0:1776189083.325550:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bf60fbf3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664284480:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189083.325572:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004ac5ba05 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664284736:12345-0@lo:38:kworker/u8:1.0 Request processed in 52us (293us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189083.325579:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9063 00000100:00100000:0.0:1776189083.325594:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bf60fbf3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664284480:12345-0@lo:400:kworker/u8:1.0 Request processed in 43us (840us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189083.325603:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9717 00000100:00100000:1.0:1776189083.325611:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e463b423 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664284480:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189083.325619:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9718 00000100:00100000:0.0:1776189083.325624:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000526ed979 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664284928:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189083.325654:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000174c978a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664285120:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189083.325675:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000526ed979 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664284928:12345-0@lo:400:kworker/u8:1.0 Request processed in 49us (721us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189083.325682:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9718 00000100:00100000:1.0:1776189083.325687:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189083.325698:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57191 00000100:00100000:0.0:1776189083.325703:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b39ea587 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664284544:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189083.325710:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189083.325723:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57192 00000100:00100000:1.0:1776189083.325728:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c510b01d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664284672:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189083.325738:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189083.325739:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b39ea587 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664284544:12345-0@lo:38:kworker/u8:1.0 Request processed in 33us (528us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189083.325747:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57191 00000100:00100000:1.0:1776189083.325770:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c510b01d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664284672:12345-0@lo:38:kworker/u8:1.0 Request processed in 40us (525us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189083.325776:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57192 00000100:00100000:1.0:1776189083.325804:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f059006a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664284864:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189083.325814:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000022f2aa27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664285056:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189083.325838:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004ae527b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664284928:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189083.326087:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664284608 00000100:00100000:3.0:1776189083.326114:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9719 00000100:00100000:3.0:1776189083.326117:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000068939cd3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664284608:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189083.326155:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000068939cd3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664284608:12345-0@lo:400:kworker/u8:1.0 Request processed in 36us (1531us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189083.326161:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9719 00000100:00100000:3.0:1776189083.326196:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664285120 00000100:00100000:3.0:1776189083.326216:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6558 00000100:00100000:3.0:1776189083.326219:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d0c1478c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664285120:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776189083.326231:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189083.326264:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d0c1478c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664285120:12345-0@lo:13:osp-pre-0-1.0 Request processed in 43us (578us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189083.326269:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6558 00000100:00100000:3.0:1776189083.326299:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000174c978a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664285120:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189083.326322:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d7babd95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664284608:0@lo:400:kworker/u8:1.0 00000100:00080000:0.0:1776189083.326386:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189083.326390:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189083.326394:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d9c93ad2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664284544:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776189083.326403:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189083.326406:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189083.326408:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000099b0e8b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664284672:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776189083.326415:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189083.326417:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189083.326419:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b7a82fb0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664284736:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189083.327108:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004d465759 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664285184:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189083.327161:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189083.328250:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664285184 00000100:00100000:0.0:1776189083.328302:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6559 00000100:00100000:0.0:1776189083.328307:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000acf34144 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664285184:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189083.328329:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189083.328390:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000acf34144 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664285184:12345-0@lo:13:osp-pre-1-1.0 Request processed in 82us (1230us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189083.328397:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6559 00000100:00100000:2.0:1776189083.329377:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004d465759 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664285184:0@lo:13:osp-pre-1-1.0 00010000:02020000:3.0:1776189083.548367:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189083.645220:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189083.804340:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189084.061118:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189084.319254:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189084.572253:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189084.669119:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189084.828276:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189085.084320:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189085.340773:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189085.596500:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189085.692456:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189085.852292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189086.108313:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189086.364282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189086.620302:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189086.722432:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189086.876903:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189087.099436:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189087.099490:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189087.099528:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189087.099576:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189087.099614:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654941760 00000100:00100000:3.0:1776189087.099614:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654941824 00000100:00100000:3.0:1776189087.099664:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654941888 00000100:00100000:1.0:1776189087.099669:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1338 00000100:00100000:1.0:1776189087.099676:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000090683405 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654941760:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189087.099686:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9720 00000100:00100000:3.0:1776189087.099691:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009820cd1e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654941824:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189087.099764:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57193 00000100:00100000:0.0:1776189087.099770:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d02fb24b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654941888:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776189087.099785:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189087.099840:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000090683405 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654941760:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 163us (350us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189087.099849:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1338 00000100:00100000:2.0:1776189087.099972:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654941952 00000100:00100000:2.0:1776189087.100007:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8377 00000100:00100000:2.0:1776189087.100012:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000067c9d69b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2279:x1862459654941952:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189087.101421:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009820cd1e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654941824:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 1725us (1988us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189087.101435:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9720 00000100:00100000:0.0:1776189087.107501:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d02fb24b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654941888:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 7722us (7917us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189087.107516:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57193 00000100:00100000:2.0:1776189087.108552:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000067c9d69b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2279:x1862459654941952:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 8535us (9020us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189087.108573:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8377 00010000:02020000:3.0:1776189087.132371:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189087.390391:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189087.420632:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776189087.420653:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000914d0a5f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664285248:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189087.420718:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189087.425471:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664285248 00000100:00100000:1.0:1776189087.425546:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6560 00000100:00100000:1.0:1776189087.425561:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002bf77dc0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664285248:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189087.425595:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189087.425673:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002bf77dc0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664285248:12345-0@lo:13:osp-pre-1-0.0 Request processed in 120us (4956us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189087.425681:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6560 00000100:00100000:0.0:1776189087.425802:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000914d0a5f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664285248:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189087.432165:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776189087.432220:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b7a82fb0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664285312:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189087.432292:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189087.432739:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664285312 00000100:00100000:3.0:1776189087.432792:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6561 00000100:00100000:3.0:1776189087.432799:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003eb43bde pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664285312:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189087.432831:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189087.432887:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003eb43bde pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664285312:12345-0@lo:13:osp-pre-0-0.0 Request processed in 87us (595us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189087.432895:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6561 00000100:00100000:2.0:1776189087.432993:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b7a82fb0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664285312:0@lo:13:osp-pre-0-0.0 00010000:02020000:3.0:1776189087.645668:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189087.740629:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189087.902900:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189088.156789:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189088.412478:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189088.444462:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776189088.444484:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004ae527b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664285440:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189088.444519:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e718565 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664285376:0@lo:13:osp-pre-1-1.0 00000100:00080000:1.0:1776189088.444676:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189088.444681:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189088.444684:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189088.444689:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9230 00000100:00080000:1.0:1776189088.444691:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776189088.444697:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189088.444714:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bc8f2779 x1862459664285632/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189088.444734:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189088.444737:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189088.444738:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189088.444741:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9230 00000100:00080000:1.0:1776189088.444743:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776189088.444746:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189088.444754:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005f37d44e x1862459664285760/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189088.444762:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189088.444764:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189088.444766:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189088.444769:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9230 00000100:00080000:1.0:1776189088.444770:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776189088.444773:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189088.444782:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000925fa05a x1862459664285824/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189088.444826:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008950b44b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664285568:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189088.444887:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bc8f2779 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664285632:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189088.448742:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189088.448800:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664285440 00000100:00100000:3.0:1776189088.448815:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ab2fefce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664285504:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189088.448856:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6562 00000100:00100000:2.0:1776189088.448863:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022f2aa27 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664285440:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189088.448869:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:2.0:1776189088.448887:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189088.448890:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000db0032a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664285952:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189088.448919:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189088.448938:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022f2aa27 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664285440:12345-0@lo:13:osp-pre-0-1.0 Request processed in 73us (202us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189088.448946:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6562 00000100:00100000:2.0:1776189088.448971:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189088.448974:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004ae527b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664285440:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189088.448998:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008ceb4933 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664285696:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189088.449077:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1776189088.449084:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d45cc7cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664285888:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189088.450834:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189088.450891:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664285952 00000100:00100000:3.0:1776189088.450934:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664285888 00000100:00100000:3.0:1776189088.450956:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8378 00000100:00100000:3.0:1776189088.450961:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f059006a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664285952:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189088.451019:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f059006a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664285952:12345-0@lo:400:kworker/u8:1.0 Request processed in 58us (2100us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189088.451029:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8378 00000100:00100000:3.0:1776189088.451043:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8379 00000100:00100000:3.0:1776189088.451046:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004ae527b6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664285888:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189088.451071:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004ae527b6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664285888:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (244us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189088.451077:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8379 00000100:00100000:3.0:1776189088.451095:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664285376 00000100:00100000:3.0:1776189088.451115:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6563 00000100:00100000:3.0:1776189088.451118:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000673b41e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664285376:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189088.451132:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189088.451159:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000673b41e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664285376:12345-0@lo:13:osp-pre-1-1.0 Request processed in 39us (2189us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189088.451164:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6563 00000100:00100000:3.0:1776189088.451208:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664285504 00000100:00100000:3.0:1776189088.451233:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1339 00000100:00100000:3.0:1776189088.451236:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b90a26ba pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664285504:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189088.451268:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b90a26ba pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664285504:12345-0@lo:400:kworker/u8:1.0 Request processed in 31us (2401us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189088.451273:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1339 00000100:00100000:3.0:1776189088.451568:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005ad21a28 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664286144:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189088.451597:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189088.451619:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664286144 00000100:00100000:3.0:1776189088.451640:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8380 00000100:00100000:3.0:1776189088.451643:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053ba7e65 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664286144:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189088.451675:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053ba7e65 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664286144:12345-0@lo:400:kworker/u8:1.0 Request processed in 30us (78us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189088.451680:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8380 00000100:00100000:3.0:1776189088.451711:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ab2fefce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664285504:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189088.451724:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000db0032a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664285952:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189088.451747:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d45cc7cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664285888:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189088.451756:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005ad21a28 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664286144:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189088.454648:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189088.454703:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064bc62e8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664286016:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189088.454746:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189088.454799:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664285568 00000100:00100000:1.0:1776189088.454859:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664286016 00000100:00100000:1.0:1776189088.454881:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9721 00000100:00100000:1.0:1776189088.454886:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005ad21a28 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664285568:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189088.454933:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005ad21a28 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664285568:12345-0@lo:400:kworker/u8:1.0 Request processed in 46us (293us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189088.454940:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9721 00000100:00100000:1.0:1776189088.454958:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9722 00000100:00100000:1.0:1776189088.454962:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d45cc7cd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664286016:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189088.454994:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d45cc7cd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664286016:12345-0@lo:400:kworker/u8:1.0 Request processed in 20us (237us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189088.455000:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9722 00000100:00100000:1.0:1776189088.455050:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008950b44b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664285568:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189088.456706:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064bc62e8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664286016:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189088.459883:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189088.459937:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005f37d44e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664285760:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189088.459979:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189088.460507:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000925fa05a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664285824:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189088.460551:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189088.460606:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664285824 00000100:00100000:0.0:1776189088.460641:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9064 00000100:00100000:0.0:1776189088.460646:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004d465759 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664285824:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189088.460662:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189088.460696:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004d465759 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664285824:12345-0@lo:38:kworker/u8:1.0 Request processed in 50us (146us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189088.460703:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9064 00000100:00080000:0.0:1776189088.460738:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189088.460742:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189088.460747:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000925fa05a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664285824:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189088.460833:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189088.460863:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005f435eca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664286080:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189088.460888:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189088.460924:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e718565 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664285376:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189088.460952:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664286080 00000100:00100000:2.0:1776189088.460980:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8381 00000100:00100000:2.0:1776189088.460983:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba2818a5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664286080:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189088.461169:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664285632 00000100:00100000:1.0:1776189088.461207:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664285760 00000100:00100000:1.0:1776189088.461219:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664285696 00000100:00100000:1.0:1776189088.461239:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9723 00000100:00100000:1.0:1776189088.461242:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b29992fd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664285696:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189088.461282:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b29992fd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664285696:12345-0@lo:400:kworker/u8:1.0 Request processed in 39us (450us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189088.461289:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9723 00000100:00100000:1.0:1776189088.461302:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57194 00000100:00100000:1.0:1776189088.461305:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000994add5b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664285632:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189088.461313:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189088.461339:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000994add5b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664285632:12345-0@lo:38:kworker/u8:1.0 Request processed in 32us (1713us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189088.461348:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57194 00000100:00100000:3.0:1776189088.461889:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57195 00000100:00100000:3.0:1776189088.461896:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a87ab8e6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664285760:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189088.461907:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189088.461944:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a87ab8e6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664285760:12345-0@lo:38:kworker/u8:1.0 Request processed in 48us (1965us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189088.461951:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57195 00000100:00080000:0.0:1776189088.462253:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189088.462259:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189088.462263:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bc8f2779 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664285632:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776189088.462273:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189088.462275:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189088.462278:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f37d44e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664285760:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189088.462523:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008ceb4933 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664285696:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189088.462881:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba2818a5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664286080:12345-0@lo:400:kworker/u8:1.0 Request processed in 1895us (1991us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189088.462891:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8381 00000100:00100000:1.0:1776189088.464459:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f435eca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664286080:0@lo:400:kworker/u8:1.0 00010000:02020000:1.0:1776189088.668352:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189088.764853:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189088.924784:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189089.180401:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189089.436308:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189089.692310:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189089.788546:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189089.948367:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189090.204332:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189090.460282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189090.716294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189090.812639:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189090.972281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189091.228625:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189091.484287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189091.740552:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189091.836493:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189091.996286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189092.226397:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189092.226510:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189092.226546:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189092.226626:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189092.227040:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654942016 00000100:00100000:2.0:1776189092.227100:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1340 00000100:00100000:2.0:1776189092.227106:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ef3ea18a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654942016:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189092.227205:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ef3ea18a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654942016:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 97us (695us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189092.227219:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1340 00000100:00100000:2.0:1776189092.227270:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654942208 00000100:00100000:2.0:1776189092.227302:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8382 00000100:00100000:2.0:1776189092.227306:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b7a82fb0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2279:x1862459654942208:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189092.227333:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b7a82fb0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2279:x1862459654942208:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 27us (787us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189092.227339:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8382 00000100:00100000:1.0:1776189092.234036:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654942144 00000100:00100000:1.0:1776189092.234119:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654942080 00000100:00100000:1.0:1776189092.234163:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9724 00000100:00100000:1.0:1776189092.234169:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000d3414bd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654942080:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189092.234691:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57196 00000100:00100000:2.0:1776189092.234697:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d92ffa01 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654942144:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776189092.234715:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189092.234788:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d92ffa01 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654942144:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 90us (8397us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189092.234796:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57196 00000100:00100000:1.0:1776189092.236807:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000d3414bd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654942080:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 2634us (10178us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189092.236818:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9724 00010000:02020000:1.0:1776189092.252348:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189092.509456:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189092.541494:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000481714e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664286208:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189092.541561:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189092.541671:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664286208 00000100:00100000:3.0:1776189092.541717:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6564 00000100:00100000:3.0:1776189092.541722:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000174c978a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664286208:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189092.541747:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189092.541786:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000174c978a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664286208:12345-0@lo:13:osp-pre-0-0.0 Request processed in 63us (226us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189092.541792:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6564 00000100:00100000:3.0:1776189092.541831:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000481714e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664286208:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189092.542007:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aafe5128 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664286272:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189092.542031:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189092.542089:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664286272 00000100:00100000:2.0:1776189092.542115:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6565 00000100:00100000:2.0:1776189092.542119:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000481714e2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664286272:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189092.542132:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189092.542163:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000481714e2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664286272:12345-0@lo:13:osp-pre-1-0.0 Request processed in 43us (132us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189092.542169:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6565 00000100:00100000:2.0:1776189092.542213:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aafe5128 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664286272:0@lo:13:osp-pre-1-0.0 00010000:02020000:1.0:1776189092.764854:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189092.861192:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189093.020516:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189093.276288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189093.532271:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189093.564502:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f4061c40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664286336:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189093.564579:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189093.564672:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664286336 00000100:00100000:0.0:1776189093.564724:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6566 00000100:00100000:0.0:1776189093.564731:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f435eca pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664286336:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776189093.564756:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189093.564811:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f435eca pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664286336:12345-0@lo:13:osp-pre-0-1.0 Request processed in 81us (233us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189093.564817:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6566 00000100:00100000:1.0:1776189093.564963:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aafe5128 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664286400:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189093.564995:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f4061c40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664286336:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189093.564996:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189093.565106:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664286400 00000100:00100000:0.0:1776189093.565136:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6567 00000100:00100000:0.0:1776189093.565140:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008ceb4933 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664286400:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189093.565151:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189093.565213:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008ceb4933 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664286400:12345-0@lo:13:osp-pre-1-1.0 Request processed in 71us (219us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189093.565219:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6567 00000100:00080000:3.0:1776189093.565316:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189093.565320:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189093.565323:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189093.565328:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9235 00000100:00080000:3.0:1776189093.565331:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00080000:3.0:1776189093.565334:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189093.565354:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000013c0eaff x1862459664286592/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189093.565372:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189093.565374:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189093.565375:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189093.565378:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9235 00000100:00080000:3.0:1776189093.565380:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00080000:3.0:1776189093.565383:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189093.565390:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000066892405 x1862459664286720/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189093.565396:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189093.565398:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189093.565399:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189093.565402:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9235 00000100:00080000:3.0:1776189093.565404:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 6s 00000100:00080000:3.0:1776189093.565406:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189093.565414:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000040cb0cca x1862459664286784/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189093.565461:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001a12eabb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664286464:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189093.565493:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189093.565506:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000051b95bd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664286912:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189093.565527:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189093.565542:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776189093.565546:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006a7d5f1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664286848:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189093.565580:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189093.565600:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000032f89d8f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664286656:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189093.565624:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189093.565634:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664286464 00000100:00100000:3.0:1776189093.565641:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1776189093.565647:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f4061c40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664286528:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189093.565670:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1341 00000100:00100000:3.0:1776189093.565674:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189093.565678:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005c923f64 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664286464:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189093.565690:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ec15c7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664286976:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189093.565713:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189093.565745:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664286656 00000100:00100000:2.0:1776189093.565746:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005c923f64 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664286464:12345-0@lo:400:kworker/u8:1.0 Request processed in 67us (251us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189093.565755:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1341 00000100:00100000:3.0:1776189093.565778:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664286528 00000100:00100000:2.0:1776189093.565791:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664286912 00000100:00100000:3.0:1776189093.565799:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664286976 00000100:00100000:3.0:1776189093.565825:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9725 00000100:00100000:2.0:1776189093.565826:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664286848 00000100:00100000:3.0:1776189093.565831:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000264516d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664286656:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189093.565869:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000264516d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664286656:12345-0@lo:400:kworker/u8:1.0 Request processed in 38us (244us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189093.565874:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9725 00000100:00100000:3.0:1776189093.565887:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9726 00000100:00100000:3.0:1776189093.565890:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000087b82cd7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664286528:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189093.565914:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000087b82cd7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664286528:12345-0@lo:400:kworker/u8:1.0 Request processed in 23us (245us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189093.565919:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9726 00000100:00100000:3.0:1776189093.565929:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9727 00000100:00100000:3.0:1776189093.565932:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005d995c27 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664286976:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189093.566017:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8383 00000100:00100000:2.0:1776189093.566021:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ae1dab54 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664286912:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189093.566055:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ae1dab54 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664286912:12345-0@lo:400:kworker/u8:1.0 Request processed in 33us (527us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189093.566062:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8383 00000100:00100000:0.0:1776189093.566067:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8384 00000100:00100000:0.0:1776189093.566075:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009fb7b0bb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664286848:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189093.566120:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aafe5128 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664286400:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189093.566202:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000013c0eaff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664286592:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189093.569741:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009fb7b0bb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664286848:12345-0@lo:400:kworker/u8:1.0 Request processed in 3657us (4152us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189093.569758:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8384 00000100:00100000:0.0:1776189093.570390:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f4061c40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664286528:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189093.570402:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000032f89d8f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664286656:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189093.571203:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001a12eabb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664286464:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189093.571645:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000051b95bd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664286912:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189093.572387:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189093.572725:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000066892405 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664286720:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189093.573247:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664286592 00000100:00100000:0.0:1776189093.573285:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57197 00000100:00100000:0.0:1776189093.573291:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000064bc62e8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664286592:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189093.573306:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189093.573345:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000064bc62e8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664286592:12345-0@lo:38:kworker/u8:1.0 Request processed in 54us (961us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189093.573351:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57197 00000100:00100000:1.0:1776189093.573452:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189093.573583:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000040cb0cca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664286784:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189093.573938:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189093.574244:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664286720 00000100:00100000:0.0:1776189093.574268:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57198 00000100:00100000:0.0:1776189093.574272:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008950b44b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664286720:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189093.574282:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189093.574309:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008950b44b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664286720:12345-0@lo:38:kworker/u8:1.0 Request processed in 36us (858us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189093.574314:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57198 00000100:00100000:1.0:1776189093.574398:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664286784 00000100:00100000:1.0:1776189093.574432:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9065 00000100:00100000:1.0:1776189093.574435:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000db0032a pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664286784:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189093.574444:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189093.574861:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000db0032a pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664286784:12345-0@lo:38:kworker/u8:1.0 Request processed in 423us (922us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189093.574868:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9065 00000100:00080000:1.0:1776189093.574902:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189093.574905:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189093.574910:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000013c0eaff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664286592:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776189093.574918:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189093.574920:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189093.574923:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000066892405 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664286720:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776189093.574929:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189093.574932:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189093.574934:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000040cb0cca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664286784:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189093.577536:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005d995c27 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664286976:12345-0@lo:400:kworker/u8:1.0 Request processed in 11596us (11816us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189093.577998:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9727 00000100:00100000:0.0:1776189093.578142:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ec15c7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664286976:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189093.582007:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006a7d5f1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664286848:0@lo:400:kworker/u8:1.0 00010000:02020000:1.0:1776189093.789089:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189093.885241:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189094.046044:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189094.300385:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189094.556300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189094.812945:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189094.908490:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189095.068288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189095.324334:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189095.580265:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189095.836426:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189095.932999:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189096.092273:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189096.348519:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189096.604313:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189096.860330:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189096.956745:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189097.116395:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189097.343270:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189097.343319:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189097.343368:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189097.343388:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654942272 00000100:00100000:0.0:1776189097.343402:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654942400 00000100:00100000:0.0:1776189097.343440:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57199 00000100:00100000:0.0:1776189097.343447:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008e6bd251 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654942400:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776189097.343466:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189097.343474:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1342 00000100:00100000:2.0:1776189097.343480:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cd00b680 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654942272:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189097.343566:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654942464 00000100:00100000:0.0:1776189097.343678:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008e6bd251 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654942400:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 86us (166us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189097.343686:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57199 00000100:00100000:1.0:1776189097.343695:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8385 00000100:00100000:1.0:1776189097.343702:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000029604218 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2278:x1862459654942464:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189097.346491:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cd00b680 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654942272:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 2850us (3068us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189097.346608:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1342 00000100:00100000:0.0:1776189097.346733:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189097.346781:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654942336 00000100:00100000:2.0:1776189097.346977:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9728 00000100:00100000:2.0:1776189097.346982:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fd2b3fb0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654942336:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189097.347293:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fd2b3fb0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654942336:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 309us (561us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189097.347300:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9728 00000100:00100000:1.0:1776189097.347360:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000029604218 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2278:x1862459654942464:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 3655us (4039us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189097.347371:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8385 00010000:02020000:1.0:1776189097.372992:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189097.628256:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189097.661746:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000079c74b68 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664287040:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189097.661825:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189097.661929:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664287040 00000100:00100000:1.0:1776189097.661981:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6568 00000100:00100000:1.0:1776189097.661987:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006e718565 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664287040:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189097.662016:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189097.662081:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006e718565 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664287040:12345-0@lo:13:osp-pre-0-0.0 Request processed in 93us (256us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189097.662088:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6568 00000100:00100000:3.0:1776189097.662139:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000079c74b68 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664287040:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189097.665496:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001a12eabb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664287104:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189097.665568:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189097.666531:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664287104 00000100:00100000:0.0:1776189097.666745:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6569 00000100:00100000:0.0:1776189097.666751:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000aafe5128 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664287104:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189097.666807:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189097.666868:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000aafe5128 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664287104:12345-0@lo:13:osp-pre-1-0.0 Request processed in 116us (1301us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189097.666874:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6569 00000100:00100000:2.0:1776189097.666927:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001a12eabb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664287104:0@lo:13:osp-pre-1-0.0 00010000:02020000:3.0:1776189097.884843:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189097.981062:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189098.140652:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189098.396339:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189098.652281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189098.684872:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f4061c40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664287168:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189098.684943:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189098.685116:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664287168 00000100:00100000:0.0:1776189098.685171:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6570 00000100:00100000:0.0:1776189098.685193:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009e6a2c02 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664287168:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189098.685224:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189098.685278:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009e6a2c02 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664287168:12345-0@lo:13:osp-pre-1-1.0 Request processed in 84us (336us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189098.685285:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6570 00000100:00100000:1.0:1776189098.685433:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f4061c40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664287168:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189098.686028:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000079c74b68 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664287232:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189098.686066:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189098.686159:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664287232 00000100:00100000:3.0:1776189098.686225:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6571 00000100:00100000:3.0:1776189098.686230:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000887d8a72 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664287232:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776189098.686246:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189098.686283:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000887d8a72 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664287232:12345-0@lo:13:osp-pre-0-1.0 Request processed in 52us (217us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189098.686289:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6571 00000100:00100000:3.0:1776189098.686333:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000079c74b68 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664287232:0@lo:13:osp-pre-0-1.0 00000100:00080000:1.0:1776189098.687851:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189098.687859:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189098.687862:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189098.687871:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9240 00000100:00080000:1.0:1776189098.687875:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776189098.687879:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189098.687897:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009fe6d891 x1862459664287424/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189098.687918:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189098.687921:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189098.687923:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189098.687926:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9240 00000100:00080000:1.0:1776189098.687928:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776189098.687938:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189098.687947:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004dd53b72 x1862459664287552/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189098.687955:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189098.687957:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189098.687959:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189098.687961:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9240 00000100:00080000:1.0:1776189098.687963:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776189098.687965:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189098.687986:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000088c2a62e x1862459664287616/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189098.688229:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009fe6d891 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664287424:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189098.688283:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189098.688321:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004dd53b72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664287552:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189098.688343:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189098.688357:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088c2a62e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664287616:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189098.688379:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189098.688413:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664287424 00000100:00100000:1.0:1776189098.688441:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664287552 00000100:00100000:1.0:1776189098.688458:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57200 00000100:00100000:1.0:1776189098.688463:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000decf8864 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664287424:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189098.688476:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189098.688512:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000decf8864 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664287424:12345-0@lo:38:kworker/u8:1.0 Request processed in 48us (229us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189098.688518:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57200 00000100:00100000:1.0:1776189098.688533:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57201 00000100:00100000:1.0:1776189098.688536:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b34cb071 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664287552:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189098.688542:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189098.688562:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b34cb071 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664287552:12345-0@lo:38:kworker/u8:1.0 Request processed in 25us (219us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189098.688567:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57201 00000100:00100000:1.0:1776189098.688586:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a03700f8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664287488:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189098.688608:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189098.688622:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1776189098.688627:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a109d002 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664287360:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189098.688646:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189098.688658:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cf6b5d65 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664287808:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189098.688676:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189098.688699:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664287488 00000100:00100000:1.0:1776189098.688720:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664287360 00000100:00100000:1.0:1776189098.688729:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664287808 00000100:00100000:1.0:1776189098.688738:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9729 00000100:00100000:1.0:1776189098.688742:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bc8f2779 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664287488:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189098.688779:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bc8f2779 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664287488:12345-0@lo:400:kworker/u8:1.0 Request processed in 36us (170us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189098.688785:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9729 00000100:00100000:1.0:1776189098.688796:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9730 00000100:00100000:1.0:1776189098.688799:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000925fa05a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664287360:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189098.688823:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000925fa05a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664287360:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (176us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189098.688828:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9730 00000100:00100000:1.0:1776189098.688838:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9731 00000100:00100000:1.0:1776189098.688841:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009ac61361 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664287808:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189098.688863:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009ac61361 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664287808:12345-0@lo:400:kworker/u8:1.0 Request processed in 21us (186us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189098.688868:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9731 00000100:00080000:1.0:1776189098.688900:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189098.689588:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189098.689594:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009fe6d891 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664287424:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776189098.689613:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189098.689616:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189098.689618:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004dd53b72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664287552:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189098.689656:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a109d002 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664287360:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189098.689665:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cf6b5d65 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664287808:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189098.689672:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a03700f8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664287488:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189098.689775:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540d168e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664287680:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189098.689815:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189098.689841:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1776189098.689846:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ab2fefce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664287296:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189098.689872:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189098.689885:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d3cfd23 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664287744:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189098.689905:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189098.689940:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664287680 00000100:00100000:3.0:1776189098.689970:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664287744 00000100:00100000:3.0:1776189098.689998:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8386 00000100:00100000:3.0:1776189098.690002:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000079c74b68 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664287680:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189098.690047:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000079c74b68 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664287680:12345-0@lo:400:kworker/u8:1.0 Request processed in 44us (232us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189098.690053:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8386 00000100:00100000:3.0:1776189098.690115:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8387 00000100:00100000:3.0:1776189098.690119:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f6664e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664287744:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189098.690146:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f6664e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664287744:12345-0@lo:400:kworker/u8:1.0 Request processed in 26us (241us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189098.690151:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8387 00000100:00100000:3.0:1776189098.690200:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d3cfd23 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664287744:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189098.690209:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540d168e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664287680:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189098.690618:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664287616 00000100:00100000:0.0:1776189098.690652:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9066 00000100:00100000:0.0:1776189098.690656:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f37d44e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664287616:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189098.690670:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189098.690717:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f37d44e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664287616:12345-0@lo:38:kworker/u8:1.0 Request processed in 61us (2338us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189098.690723:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9066 00000100:00100000:0.0:1776189098.690744:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664287296 00000100:00100000:0.0:1776189098.690768:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1343 00000100:00100000:0.0:1776189098.690772:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f6dc323d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664287296:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189098.690801:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f6dc323d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664287296:12345-0@lo:400:kworker/u8:1.0 Request processed in 29us (930us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189098.690807:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1343 00000100:00080000:3.0:1776189098.692215:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189098.692224:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189098.692231:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088c2a62e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664287616:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189098.692269:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ab2fefce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664287296:0@lo:400:kworker/u8:1.0 00010000:02020000:3.0:1776189098.909205:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189099.004412:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189099.165220:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189099.420287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189099.677752:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189099.932300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189100.031360:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189100.188290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189100.448373:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189100.700758:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189100.957248:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189101.054026:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189101.212302:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189101.468282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189101.724337:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189101.887117:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189101.888369:0:6249:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654942528 00000100:00100000:1.0:1776189101.888540:0:6249:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 2722 00000100:00100000:1.0:1776189101.888547:0:6249:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f4061c40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+8:2279:x1862459654942528:12345-192.168.203.25@tcp:103:ldlm_bl_01.0 00010000:00010000:1.0:1776189101.888564:0:6249:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00002000:00010000:1.0:1776189101.888582:0:6249:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x100000000:0x1684:0x0] updating lvb size from disk: 0 -> 2123 00002000:00010000:1.0:1776189101.888585:0:6249:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x100000000:0x1684:0x0] updating lvb mtime from disk: 0 -> 1776185656 00002000:00010000:1.0:1776189101.888587:0:6249:0:(ofd_lvb.c:343:ofd_lvbo_update()) res: [0x100000000:0x1684:0x0] updating lvb ctime from disk: 0 -> 1776185656 00002000:00010000:1.0:1776189101.888589:0:6249:0:(ofd_lvb.c:349:ofd_lvbo_update()) res: [0x100000000:0x1684:0x0] updating lvb blocks from disk: 0 -> 8 00010000:00010000:1.0:1776189101.888662:0:6249:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8b7fe9f/0xed5c52a46e4396da lrc: 0/0,0 mode: --/PW res: [0x1684:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801000020000 nid: 192.168.203.25@tcp remote: 0xa474374c00046469 expref: 8 pid: 44523 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1776189101.888692:0:6249:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x100000000:0x160d:0x0] updating lvb size from disk: 0 -> 4096 00002000:00010000:1.0:1776189101.888694:0:6249:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x100000000:0x160d:0x0] updating lvb mtime from disk: 0 -> 1776185428 00002000:00010000:1.0:1776189101.888696:0:6249:0:(ofd_lvb.c:343:ofd_lvbo_update()) res: [0x100000000:0x160d:0x0] updating lvb ctime from disk: 0 -> 1776185428 00002000:00010000:1.0:1776189101.888697:0:6249:0:(ofd_lvb.c:349:ofd_lvbo_update()) res: [0x100000000:0x160d:0x0] updating lvb blocks from disk: 0 -> 8 00010000:00010000:1.0:1776189101.888798:0:6249:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dbb9f8d8/0xed5c52a46e4379de lrc: 0/0,0 mode: --/PW res: [0x160d:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0xa474374c0004608e expref: 7 pid: 44535 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1776189101.888836:0:6249:0:(ofd_lvb.c:170:ofd_lvbo_init()) res: [0x100000000:0x15a2:0x0] initial LVB size: 4096, mtime: 0x69de6f4e, atime: 0x0, ctime: 0x69de6f4e, blocks: 0x8 00010000:00010000:1.0:1776189101.888850:0:6249:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea28ce0d/0xed5c52a46e436069 lrc: 0/0,0 mode: --/PW res: [0x15a2:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0xa474374c00045dee expref: 6 pid: 170843 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776189101.888862:0:6249:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1776189101.890071:0:6249:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f4061c40 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+5:2279:x1862459654942528:12345-192.168.203.25@tcp:103:ldlm_bl_01.0 Request processed in 1517us (2957us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189101.890086:0:6249:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 2722 00010000:02020000:3.0:1776189101.980339:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189102.076422:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189102.236324:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189102.459537:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189102.459581:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189102.459623:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189102.459706:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654942784 00000100:00100000:2.0:1776189102.459713:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654942656 00000100:00100000:1.0:1776189102.459756:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8388 00000100:00100000:2.0:1776189102.459756:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9732 00000100:00100000:1.0:1776189102.459763:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001a12eabb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+5:2278:x1862459654942784:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189102.459763:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000511337f8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654942656:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189102.460011:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000511337f8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654942656:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 246us (386us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189102.460019:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9732 00000100:00100000:3.0:1776189102.460244:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654942592 00000100:00100000:3.0:1776189102.460281:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1344 00000100:00100000:3.0:1776189102.460286:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000078287bc4 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654942592:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189102.460705:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001a12eabb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+5:2278:x1862459654942784:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 941us (1123us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189102.460714:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8388 00000100:00100000:3.0:1776189102.461327:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000078287bc4 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654942592:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 1038us (1797us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189102.461337:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1344 00000100:00100000:3.0:1776189102.461810:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189102.462886:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654942720 00000100:00100000:2.0:1776189102.463669:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57202 00000100:00100000:2.0:1776189102.463677:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000058a2b052 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654942720:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776189102.463696:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189102.464298:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000058a2b052 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654942720:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 618us (2487us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189102.464308:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57202 00010000:02020000:3.0:1776189102.492416:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189102.749074:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189102.781438:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776189102.783320:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776189102.783344:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f0b13a19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664287936:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189102.783411:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189102.784448:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664287936 00000100:00100000:3.0:1776189102.784507:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6572 00000100:00100000:3.0:1776189102.784513:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b1d7bdea pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664287936:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189102.784542:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189102.784597:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b1d7bdea pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664287936:12345-0@lo:13:osp-pre-0-0.0 Request processed in 84us (1186us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189102.784605:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6572 00000100:00100000:1.0:1776189102.784771:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f0b13a19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664287936:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189102.786713:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064cbacbf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664287872:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189102.786784:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189102.787027:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664287872 00000100:00100000:1.0:1776189102.787076:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6573 00000100:00100000:1.0:1776189102.787081:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000db8bee4c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664287872:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189102.787108:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189102.787155:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000db8bee4c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664287872:12345-0@lo:13:osp-pre-1-0.0 Request processed in 73us (372us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189102.787161:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6573 00000100:00100000:2.0:1776189102.788012:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064cbacbf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664287872:0@lo:13:osp-pre-1-0.0 00010000:02020000:3.0:1776189103.004310:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189103.100391:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189103.260727:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189103.516300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189103.772324:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:1.0:1776189103.804430:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189103.804437:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189103.804440:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189103.804448:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9245 00000100:00080000:1.0:1776189103.804452:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:1.0:1776189103.804474:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189103.804494:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000c494ab9 x1862459664288128/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189103.804515:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189103.804518:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189103.804520:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189103.804522:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9245 00000100:00080000:1.0:1776189103.804524:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00080000:1.0:1776189103.804527:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189103.804537:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000229fc20a x1862459664288256/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189103.804543:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189103.804545:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189103.804547:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189103.804549:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9245 00000100:00080000:1.0:1776189103.804550:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00080000:1.0:1776189103.804553:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189103.804561:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000099b0e8b4 x1862459664288320/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189103.804617:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e5080bed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664288192:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189103.804690:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189103.804715:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005c66c1fe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664288576:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189103.804738:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189103.804773:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1776189103.804778:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe5f85f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664288064:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189103.804798:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189103.804812:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e463b423 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664288512:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189103.804828:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189103.804860:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664288192 00000100:00100000:1.0:1776189103.804912:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664288064 00000100:00100000:1.0:1776189103.804928:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664288512 00000100:00100000:1.0:1776189103.804945:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9733 00000100:00100000:1.0:1776189103.804950:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000088c2a62e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664288192:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189103.804995:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000088c2a62e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664288192:12345-0@lo:400:kworker/u8:0.0 Request processed in 44us (307us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189103.805002:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9733 00000100:00100000:1.0:1776189103.805016:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9734 00000100:00100000:1.0:1776189103.805019:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005d3cfd23 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664288064:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189103.805038:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005d3cfd23 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664288064:12345-0@lo:400:kworker/u8:0.0 Request processed in 19us (241us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189103.805043:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9734 00000100:00100000:1.0:1776189103.805051:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9735 00000100:00100000:1.0:1776189103.805054:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a03700f8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664288512:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189103.805070:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a03700f8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664288512:12345-0@lo:400:kworker/u8:0.0 Request processed in 16us (242us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189103.805074:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9735 00000100:00100000:1.0:1776189103.805119:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664288576 00000100:00100000:1.0:1776189103.805144:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8389 00000100:00100000:1.0:1776189103.805147:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000540d168e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664288576:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189103.805211:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000540d168e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664288576:12345-0@lo:400:kworker/u8:0.0 Request processed in 62us (472us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189103.805217:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8389 00000100:00100000:1.0:1776189103.805251:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe5f85f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664288064:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189103.805259:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e463b423 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664288512:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189103.805267:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e5080bed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664288192:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189103.805273:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005c66c1fe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664288576:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189103.805360:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a0c7825a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664288000:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189103.805395:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189103.805419:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d7babd95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664288448:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189103.805441:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189103.805454:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:2.0:1776189103.805459:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d9c93ad2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664288384:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189103.805479:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189103.805491:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ab2fefce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664288640:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189103.805509:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189103.805534:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664288448 00000100:00100000:2.0:1776189103.805573:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664288384 00000100:00100000:2.0:1776189103.805589:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664288640 00000100:00100000:2.0:1776189103.805602:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8390 00000100:00100000:2.0:1776189103.805605:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c0205bdc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664288448:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189103.805634:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c0205bdc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664288448:12345-0@lo:400:kworker/u8:0.0 Request processed in 29us (194us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189103.805639:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8390 00000100:00100000:2.0:1776189103.805650:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8391 00000100:00100000:2.0:1776189103.805653:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba2818a5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664288384:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189103.805673:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba2818a5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664288384:12345-0@lo:400:kworker/u8:0.0 Request processed in 20us (195us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189103.805678:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8391 00000100:00100000:2.0:1776189103.805686:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8392 00000100:00100000:2.0:1776189103.805689:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000914d0a5f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664288640:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189103.805709:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000914d0a5f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664288640:12345-0@lo:400:kworker/u8:0.0 Request processed in 19us (200us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189103.805713:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8392 00000100:00100000:2.0:1776189103.805746:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d9c93ad2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664288384:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189103.805755:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ab2fefce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664288640:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189103.805762:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d7babd95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664288448:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189103.805923:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003614bde1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664288704:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189103.805948:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189103.806169:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664288704 00000100:00100000:0.0:1776189103.806249:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6574 00000100:00100000:0.0:1776189103.806253:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003fb63a1c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664288704:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189103.806268:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189103.806301:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003fb63a1c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664288704:12345-0@lo:13:osp-pre-1-1.0 Request processed in 47us (352us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189103.806307:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6574 00000100:00100000:2.0:1776189103.806408:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003614bde1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664288704:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189103.806515:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c494ab9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664288128:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189103.806546:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189103.806561:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000229fc20a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664288256:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189103.806582:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189103.806593:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000099b0e8b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664288320:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189103.806640:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189103.806728:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664288128 00000100:00100000:0.0:1776189103.806759:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664288256 00000100:00100000:0.0:1776189103.806775:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57203 00000100:00100000:0.0:1776189103.806779:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f0b13a19 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664288128:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189103.806792:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189103.806825:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f0b13a19 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664288128:12345-0@lo:38:kworker/u8:0.0 Request processed in 45us (279us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189103.806830:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57203 00000100:00100000:0.0:1776189103.806841:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57204 00000100:00100000:0.0:1776189103.806844:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c06060ec pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664288256:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189103.806851:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189103.806872:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c06060ec pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664288256:12345-0@lo:38:kworker/u8:0.0 Request processed in 27us (290us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189103.806877:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57204 00000100:00100000:0.0:1776189103.807004:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f4f823f8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664288768:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189103.807032:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189103.807107:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664288768 00000100:00100000:2.0:1776189103.807141:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6575 00000100:00100000:2.0:1776189103.807145:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000030d1626f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664288768:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189103.807158:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189103.807228:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000030d1626f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664288768:12345-0@lo:13:osp-pre-0-1.0 Request processed in 82us (196us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189103.807234:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6575 00000100:00100000:0.0:1776189103.807309:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f4f823f8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664288768:0@lo:13:osp-pre-0-1.0 00000100:00080000:3.0:1776189103.807388:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189103.807392:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189103.807396:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c494ab9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664288128:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776189103.807404:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189103.807406:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189103.807408:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000229fc20a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664288256:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189103.811147:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664288000 00000100:00100000:1.0:1776189103.811271:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1345 00000100:00100000:1.0:1776189103.811278:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003afac4ae pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664288000:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189103.811385:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003afac4ae pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664288000:12345-0@lo:400:kworker/u8:0.0 Request processed in 106us (5989us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189103.811393:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1345 00000100:00100000:1.0:1776189103.811436:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664288320 00000100:00100000:1.0:1776189103.811455:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9067 00000100:00100000:1.0:1776189103.811459:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001d029ec5 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664288320:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189103.811474:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189103.811509:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001d029ec5 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664288320:12345-0@lo:38:kworker/u8:0.0 Request processed in 49us (4870us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189103.811516:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9067 00000100:00080000:3.0:1776189103.812033:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189103.812039:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189103.812044:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000099b0e8b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664288320:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189103.812083:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a0c7825a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664288000:0@lo:400:kworker/u8:0.0 00010000:02020000:3.0:1776189104.028322:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189104.126902:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189104.284282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189104.541073:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189104.796761:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189105.052338:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189105.150081:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189105.312090:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189105.564352:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189105.820380:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189106.076518:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189106.176392:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189106.332378:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189106.588280:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189106.844355:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189107.100312:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189107.196414:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189107.356303:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189107.581322:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189107.581443:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189107.581483:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189107.581585:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654942912 00000100:00100000:3.0:1776189107.581650:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9736 00000100:00100000:3.0:1776189107.581657:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000055c776dd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654942912:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189107.581731:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000055c776dd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654942912:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 74us (248us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189107.581740:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9736 00000100:00100000:1.0:1776189107.581909:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654943040 00000100:00100000:1.0:1776189107.581949:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8393 00000100:00100000:1.0:1776189107.581953:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cc7962a4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+5:2279:x1862459654943040:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189107.581986:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cc7962a4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+5:2279:x1862459654943040:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 32us (544us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189107.581992:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8393 00000100:00100000:0.0:1776189107.585409:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189107.585595:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654942976 00000100:00100000:3.0:1776189107.585643:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57205 00000100:00100000:3.0:1776189107.585650:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000852f310a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654942976:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:3.0:1776189107.585668:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776189107.585724:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000852f310a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654942976:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 74us (323us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189107.585731:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57205 00000100:00100000:2.0:1776189107.585998:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654942848 00000100:00100000:2.0:1776189107.586041:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1346 00000100:00100000:2.0:1776189107.586046:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000921cb3a0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654942848:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189107.586125:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000921cb3a0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654942848:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 67us (4800us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189107.586132:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1346 00010000:02020000:1.0:1776189107.612284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189107.868460:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189107.902045:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002e574ef8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664288832:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189107.902216:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776189107.902222:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189107.902234:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000960dca04 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664288896:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189107.902267:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189107.902296:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664288832 00000100:00100000:1.0:1776189107.902347:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664288896 00000100:00100000:1.0:1776189107.902384:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6576 00000100:00100000:1.0:1776189107.902389:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cf6b5d65 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664288832:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189107.902417:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189107.902456:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cf6b5d65 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664288832:12345-0@lo:13:osp-pre-1-0.0 Request processed in 65us (236us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189107.902463:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6576 00000100:00100000:1.0:1776189107.902481:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6577 00000100:00100000:1.0:1776189107.902485:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000f2a9137 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664288896:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189107.902500:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189107.902526:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000f2a9137 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664288896:12345-0@lo:13:osp-pre-0-0.0 Request processed in 40us (260us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189107.902532:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6577 00000100:00100000:3.0:1776189107.902579:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000960dca04 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664288896:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189107.902850:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002e574ef8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664288832:0@lo:13:osp-pre-1-0.0 00010000:02020000:1.0:1776189108.124838:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189108.220834:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189108.380362:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189108.636293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189108.892376:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189108.924359:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776189108.924382:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009be3a12f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664289024:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189108.924457:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189108.924518:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f4f823f8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664288960:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189108.924543:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189108.924580:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664289024 00000100:00100000:0.0:1776189108.924618:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664288960 00000100:00100000:0.0:1776189108.924646:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6578 00000100:00100000:0.0:1776189108.924652:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000381790a0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664289024:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189108.924678:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189108.924716:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000381790a0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664289024:12345-0@lo:13:osp-pre-1-1.0 Request processed in 63us (260us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189108.924723:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6578 00000100:00100000:0.0:1776189108.924741:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6579 00000100:00100000:0.0:1776189108.924744:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002e574ef8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664288960:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776189108.924755:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189108.924782:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002e574ef8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664288960:12345-0@lo:13:osp-pre-0-1.0 Request processed in 36us (239us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189108.924788:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6579 00000100:00100000:0.0:1776189108.924831:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f4f823f8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664288960:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189108.924863:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009be3a12f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664289024:0@lo:13:osp-pre-1-1.0 00000100:00080000:1.0:1776189108.924948:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189108.924953:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189108.924957:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1776189108.925003:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776189108.925015:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004dd53b72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664289152:0@lo:400:kworker/u8:0.0 00000100:00080000:1.0:1776189108.925046:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9250 00000100:00100000:3.0:1776189108.925047:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776189108.925053:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189108.925058:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189108.925079:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009fe6d891 x1862459664289216/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189108.925136:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664289152 00000100:00100000:2.0:1776189108.925195:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9737 00000100:00100000:2.0:1776189108.925200:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000960dca04 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664289152:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189108.925245:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000960dca04 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664289152:12345-0@lo:400:kworker/u8:0.0 Request processed in 44us (199us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189108.925252:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9737 00000100:00100000:2.0:1776189108.925279:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004dd53b72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664289152:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189108.925302:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009fe6d891 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664289216:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189108.925327:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189108.925349:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664289216 00000100:00100000:2.0:1776189108.925369:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57206 00000100:00100000:2.0:1776189108.925372:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f4f823f8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664289216:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189108.925385:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189108.925415:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f4f823f8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664289216:12345-0@lo:38:kworker/u8:0.0 Request processed in 41us (87us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189108.925420:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57206 00000100:00080000:2.0:1776189108.925446:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189108.925449:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189108.925452:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009fe6d891 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664289216:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776189108.925483:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189108.925487:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189108.925489:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189108.925493:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9250 00000100:00080000:1.0:1776189108.925496:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1776189108.925499:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ab2fefce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664289280:0@lo:400:kworker/u8:0.0 00000100:00080000:1.0:1776189108.925501:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189108.925528:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189108.925553:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664289280 00000100:00100000:2.0:1776189108.925574:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9738 00000100:00100000:2.0:1776189108.925578:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000023121a7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664289280:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189108.925609:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000023121a7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664289280:12345-0@lo:400:kworker/u8:0.0 Request processed in 30us (81us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189108.925614:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9738 00000100:00100000:1.0:1776189108.925628:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009fe6d891 x1862459664289344/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189108.925641:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ab2fefce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664289280:0@lo:400:kworker/u8:0.0 00000100:00080000:1.0:1776189108.925645:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189108.925649:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189108.925651:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189108.925656:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9250 00000100:00080000:1.0:1776189108.925658:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189108.925662:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189108.925673:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009fe6d891 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664289344:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189108.925677:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004dd53b72 x1862459664289408/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189108.925702:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189108.925719:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004dd53b72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664289408:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189108.925742:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189108.925764:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664289344 00000100:00100000:2.0:1776189108.925780:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57207 00000100:00100000:2.0:1776189108.925783:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000091ef61c6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664289344:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189108.925791:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189108.925817:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000091ef61c6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664289344:12345-0@lo:38:kworker/u8:0.0 Request processed in 33us (116us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189108.925824:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57207 00000100:00080000:2.0:1776189108.925845:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189108.925848:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189108.925851:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009fe6d891 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664289344:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189108.925929:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776189108.926064:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a0c7825a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664289600:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189108.926103:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189108.926203:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664289408 00000100:00100000:0.0:1776189108.926235:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9068 00000100:00100000:0.0:1776189108.926238:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fe93ff8e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664289408:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189108.926247:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189108.926285:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ab2fefce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664289472:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189108.926291:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fe93ff8e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664289408:12345-0@lo:38:kworker/u8:0.0 Request processed in 50us (547us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189108.926299:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9068 00000100:00100000:1.0:1776189108.926322:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189108.926334:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a109d002 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664289088:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189108.926361:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664289472 00000100:00100000:0.0:1776189108.926369:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189108.926391:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009fe6d891 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664289536:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189108.926396:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8394 00000100:00100000:1.0:1776189108.926401:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000099b0e8b4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664289472:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189108.926417:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189108.926450:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000099b0e8b4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664289472:12345-0@lo:400:kworker/u8:0.0 Request processed in 48us (128us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189108.926455:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664289536 00000100:00100000:1.0:1776189108.926458:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8394 00000100:00100000:1.0:1776189108.926478:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8395 00000100:00100000:1.0:1776189108.926483:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009be3a12f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664289536:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189108.926491:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664289088 00000100:00100000:0.0:1776189108.926517:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1347 00000100:00100000:0.0:1776189108.926522:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008da73e5e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664289088:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189108.926532:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009be3a12f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664289536:12345-0@lo:400:kworker/u8:0.0 Request processed in 47us (114us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189108.926539:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8395 00000100:00100000:0.0:1776189108.926559:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008da73e5e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664289088:12345-0@lo:400:kworker/u8:0.0 Request processed in 35us (190us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189108.926567:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1347 00000100:00100000:1.0:1776189108.926577:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ab2fefce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664289472:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189108.926605:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a109d002 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664289088:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189108.926615:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009fe6d891 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664289536:0@lo:400:kworker/u8:0.0 00000100:00080000:2.0:1776189108.926730:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189108.926734:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189108.926737:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004dd53b72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664289408:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189108.927144:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664289600 00000100:00100000:3.0:1776189108.927202:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9739 00000100:00100000:3.0:1776189108.927207:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001ae8c4d3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664289600:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189108.927269:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001ae8c4d3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664289600:12345-0@lo:400:kworker/u8:0.0 Request processed in 61us (1166us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189108.927275:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9739 00000100:00100000:0.0:1776189108.930826:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a0c7825a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664289600:0@lo:400:kworker/u8:0.0 00010000:02020000:1.0:1776189109.148355:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189109.244405:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189109.404296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189109.660959:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189109.916261:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189110.175640:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189110.269873:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189110.429219:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189110.684297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189110.940598:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189111.196289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189111.292406:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189111.452303:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189111.709254:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189111.964350:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189112.220295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189112.316662:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189112.476746:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189112.704092:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189112.704160:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189112.704247:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189112.704527:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189112.704567:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654943168 00000100:00100000:0.0:1776189112.704617:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654943104 00000100:00100000:3.0:1776189112.704661:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654943232 00000100:00100000:0.0:1776189112.704676:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1348 00000100:00100000:0.0:1776189112.704687:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c3d39af pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654943104:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189112.704689:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9740 00000100:00100000:3.0:1776189112.704697:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000036a45663 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654943168:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189112.704706:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654943296 00000100:00100000:2.0:1776189112.704757:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8396 00000100:00100000:2.0:1776189112.704769:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c7c1f715 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+5:2279:x1862459654943296:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189112.704769:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000036a45663 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654943168:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 67us (603us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189112.704783:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9740 00000100:00100000:3.0:1776189112.704803:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57208 00000100:00100000:3.0:1776189112.704807:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000114609f6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654943232:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:3.0:1776189112.704822:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189112.704840:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c3d39af pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654943104:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 155us (755us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189112.704849:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1348 00000100:00100000:3.0:1776189112.704856:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000114609f6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654943232:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 46us (608us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189112.704863:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57208 00000100:00100000:2.0:1776189112.704902:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c7c1f715 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+5:2279:x1862459654943296:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 138us (456us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189112.704910:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8396 00010000:02020000:1.0:1776189112.732390:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189112.988319:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189113.020543:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ce461df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664289664:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189113.020665:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189113.020910:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776189113.020911:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664289664 00000100:00100000:0.0:1776189113.020923:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006392c95c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664289728:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189113.020954:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189113.020968:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664289728 00000100:00100000:3.0:1776189113.020995:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6580 00000100:00100000:3.0:1776189113.021010:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003614bde1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664289664:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189113.021053:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189113.021103:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003614bde1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664289664:12345-0@lo:13:osp-pre-0-0.0 Request processed in 92us (441us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189113.021110:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6580 00000100:00100000:3.0:1776189113.021127:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6581 00000100:00100000:3.0:1776189113.021130:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000064cbacbf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664289728:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776189113.021144:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189113.021169:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000064cbacbf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664289728:12345-0@lo:13:osp-pre-1-0.0 Request processed in 38us (216us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189113.021193:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6581 00000100:00100000:1.0:1776189113.021269:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ce461df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664289664:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189113.028396:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006392c95c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664289728:0@lo:13:osp-pre-1-0.0 00010000:02020000:1.0:1776189113.244772:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189113.340535:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189113.500886:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189113.756318:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189114.012312:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:1.0:1776189114.044396:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189114.044403:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1776189114.044408:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f326e309 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664289856:0@lo:13:osp-pre-1-1.0 00000100:00080000:1.0:1776189114.044408:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189114.044416:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9255 00000100:00080000:1.0:1776189114.044420:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189114.044426:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189114.044480:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189114.044498:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d7babd95 x1862459664289984/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189114.044522:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189114.044525:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1776189114.044527:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00080000:1.0:1776189114.044527:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189114.044533:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9255 00000100:00080000:1.0:1776189114.044535:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1776189114.044536:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c494ab9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664289920:0@lo:400:kworker/u8:0.0 00000100:00080000:1.0:1776189114.044540:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189114.044564:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005c66c1fe x1862459664290112/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189114.044571:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776189114.044578:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189114.044581:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189114.044582:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189114.044585:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9255 00000100:00080000:1.0:1776189114.044587:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189114.044590:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189114.044599:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e5080bed x1862459664290176/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189114.044616:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664289920 00000100:00100000:1.0:1776189114.044684:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d9c93ad2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664290048:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189114.044689:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9741 00000100:00100000:0.0:1776189114.044696:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c6f4c7a3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664289920:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189114.044734:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189114.044755:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c6f4c7a3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664289920:12345-0@lo:400:kworker/u8:0.0 Request processed in 57us (186us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189114.044759:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004dd53b72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664290432:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189114.044765:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9741 00000100:00100000:0.0:1776189114.044792:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664290048 00000100:00100000:1.0:1776189114.044795:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189114.044818:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1776189114.044819:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9742 00000100:00100000:0.0:1776189114.044826:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009fe6d891 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664290048:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189114.044826:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000229fc20a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664289792:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189114.045207:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d7babd95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664289984:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189114.045295:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003258ba66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664290496:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189114.048058:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189114.048092:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189114.048130:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e463b423 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664290240:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189114.048130:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005c66c1fe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664290112:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189114.048199:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189114.048204:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189114.048405:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e5080bed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664290176:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189114.048436:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664289984 00000100:00100000:1.0:1776189114.048470:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664290112 00000100:00100000:1.0:1776189114.048484:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57209 00000100:00100000:1.0:1776189114.048490:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bde4c712 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664289984:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189114.048508:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189114.048547:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bde4c712 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664289984:12345-0@lo:38:kworker/u8:0.0 Request processed in 57us (457us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189114.048554:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57209 00000100:00100000:2.0:1776189114.048574:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189114.048577:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57210 00000100:00100000:1.0:1776189114.048582:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006392c95c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664290112:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189114.048590:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189114.048620:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006392c95c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664290112:12345-0@lo:38:kworker/u8:0.0 Request processed in 37us (418us total) trans 0 rc -19/-19 00000100:00080000:2.0:1776189114.048620:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189114.048626:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189114.048630:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57210 00000100:00100000:2.0:1776189114.048633:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d7babd95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664289984:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776189114.048647:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189114.048650:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189114.048654:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664290432 00000100:00100000:2.0:1776189114.048658:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005c66c1fe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664290112:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189114.048687:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664290240 00000100:00100000:1.0:1776189114.048707:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8397 00000100:00100000:1.0:1776189114.048710:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a109d002 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664290432:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189114.048755:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a109d002 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664290432:12345-0@lo:400:kworker/u8:0.0 Request processed in 44us (3962us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189114.048765:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8397 00000100:00100000:1.0:1776189114.048776:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8398 00000100:00100000:1.0:1776189114.048779:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ab2fefce pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664290240:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189114.049638:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a0c7825a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664290368:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189114.049683:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189114.049710:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776189114.049716:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe5f85f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664290304:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189114.049873:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189114.049924:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664290176 00000100:00100000:2.0:1776189114.049953:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9069 00000100:00100000:2.0:1776189114.049957:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001b0d11fc pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664290176:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189114.049970:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189114.050091:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001b0d11fc pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664290176:12345-0@lo:38:kworker/u8:0.0 Request processed in 132us (1519us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189114.050098:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9069 00000100:00100000:2.0:1776189114.050122:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664290304 00000100:00100000:2.0:1776189114.050143:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8399 00000100:00100000:2.0:1776189114.050147:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009b5045f9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664290304:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189114.050266:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009b5045f9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664290304:12345-0@lo:400:kworker/u8:0.0 Request processed in 117us (393us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189114.050272:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8399 00000100:00100000:2.0:1776189114.050303:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe5f85f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664290304:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189114.050335:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c494ab9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664289920:0@lo:400:kworker/u8:0.0 00000100:00080000:2.0:1776189114.050550:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189114.050554:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189114.050557:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e5080bed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664290176:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189114.050581:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664289856 00000100:00100000:2.0:1776189114.050615:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6582 00000100:00100000:2.0:1776189114.050619:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003a715368 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664289856:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189114.050638:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189114.050672:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003a715368 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664289856:12345-0@lo:13:osp-pre-1-1.0 Request processed in 53us (6194us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189114.050678:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6582 00000100:00100000:2.0:1776189114.050705:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f326e309 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664289856:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189114.050951:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189114.050969:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664290368 00000100:00100000:3.0:1776189114.050995:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664290496 00000100:00100000:2.0:1776189114.051001:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9743 00000100:00100000:2.0:1776189114.051005:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ddffdaa5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664290368:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189114.051026:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6583 00000100:00100000:3.0:1776189114.051032:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a38efcb6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664290496:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189114.051048:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ddffdaa5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664290368:12345-0@lo:400:kworker/u8:0.0 Request processed in 40us (1365us total) trans 0 rc 0/0 00002000:00000024:3.0:1776189114.051051:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189114.051060:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9743 00000100:00100000:2.0:1776189114.051092:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a0c7825a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664290368:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189114.051123:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004dd53b72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664290432:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189114.051154:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003258ba66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664290496:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189114.051224:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664289792 00000100:00100000:2.0:1776189114.051246:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1349 00000100:00100000:2.0:1776189114.051249:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e858e44a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664289792:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189114.055085:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e858e44a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664289792:12345-0@lo:400:kworker/u8:0.0 Request processed in 3823us (7023us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189114.055103:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1349 00000100:00100000:2.0:1776189114.055332:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000229fc20a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664289792:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189114.057237:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009fe6d891 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664290048:12345-0@lo:400:kworker/u8:0.0 Request processed in 12405us (12497us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189114.057251:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9742 00000100:00100000:2.0:1776189114.057344:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d9c93ad2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664290048:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189114.058086:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ab2fefce pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664290240:12345-0@lo:400:kworker/u8:0.0 Request processed in 9302us (9884us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189114.058100:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8398 00000100:00100000:2.0:1776189114.061129:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e463b423 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664290240:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189114.063425:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a38efcb6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664290496:12345-0@lo:13:osp-pre-0-1.0 Request processed in 12384us (12467us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189114.063442:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6583 00010000:02020000:1.0:1776189114.269077:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189114.365392:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189114.524383:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189114.780732:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189115.036507:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189115.293586:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189115.388444:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189115.550134:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189115.804313:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189116.060561:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189116.317673:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189116.412877:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189116.572723:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189116.828303:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189117.084289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189117.341031:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189117.436487:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189117.597443:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189117.831370:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189117.831418:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189117.831450:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189117.831487:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189117.831492:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654943360 00000100:00100000:0.0:1776189117.831528:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654943424 00000100:00100000:2.0:1776189117.831548:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1350 00000100:00100000:0.0:1776189117.831552:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654943488 00000100:00100000:2.0:1776189117.831554:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ff8d0a76 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654943360:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776189117.831583:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9744 00000100:00100000:0.0:1776189117.831587:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000056bf1b01 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654943424:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1776189117.831607:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654943552 00000100:00100000:2.0:1776189117.831622:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ff8d0a76 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654943360:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 66us (203us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189117.831631:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1350 00000100:00100000:1.0:1776189117.831647:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8400 00000100:00100000:1.0:1776189117.831653:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000279e850a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+5:2281:x1862459654943552:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1776189117.831659:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57211 00000100:00100000:3.0:1776189117.831665:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000068c80116 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654943488:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:3.0:1776189117.831680:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189117.836128:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000056bf1b01 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654943424:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 4530us (4757us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189117.836155:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9744 00000100:00100000:1.0:1776189117.838451:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000279e850a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+5:2281:x1862459654943552:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 6789us (6992us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189117.838470:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8400 00000100:00100000:3.0:1776189117.839088:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000068c80116 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654943488:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 7418us (7598us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189117.839099:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57211 00010000:02020000:1.0:1776189117.852650:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189118.108354:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189118.140754:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ce461df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664290560:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189118.140983:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003258ba66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664290624:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189118.141045:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189118.141151:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664290624 00000100:00100000:3.0:1776189118.141227:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6584 00000100:00100000:3.0:1776189118.141233:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004dd53b72 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664290624:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776189118.141262:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189118.141309:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004dd53b72 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664290624:12345-0@lo:13:osp-pre-1-0.0 Request processed in 75us (265us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189118.141317:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6584 00000100:00100000:0.0:1776189118.141621:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189118.141698:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664290560 00000100:00100000:1.0:1776189118.141707:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003258ba66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664290624:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189118.141740:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6585 00000100:00100000:2.0:1776189118.141745:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009257ba59 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664290560:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189118.141762:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189118.142061:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009257ba59 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664290560:12345-0@lo:13:osp-pre-0-0.0 Request processed in 315us (441us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189118.142068:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6585 00000100:00100000:0.0:1776189118.142136:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ce461df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664290560:0@lo:13:osp-pre-0-0.0 00010000:02020000:1.0:1776189118.365330:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189118.463630:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189118.620336:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189118.876298:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189119.164430:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189119.164624:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776189119.164644:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003258ba66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664290688:0@lo:13:osp-pre-0-1.0 00000100:00080000:0.0:1776189119.164939:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189119.164944:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189119.164948:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189119.164953:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9261 00000100:00080000:0.0:1776189119.164956:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776189119.164960:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189119.164976:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e0967c51 x1862459664290944/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189119.165006:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189119.165009:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189119.165010:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189119.165013:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9261 00000100:00080000:0.0:1776189119.165015:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776189119.165017:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189119.165026:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000f61bb34 x1862459664291072/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189119.165033:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189119.165035:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189119.165036:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189119.165039:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9261 00000100:00080000:0.0:1776189119.165040:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776189119.165043:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189119.165100:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e071bb1b x1862459664291136/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189119.165152:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000071c1b92f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664290816:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189119.165287:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e0967c51 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664290944:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189119.165353:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000051b336a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664291008:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189119.168088:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189119.168138:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009fddac3a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664291264:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189119.169281:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189119.169316:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1776189119.169324:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e5ef59b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664290880:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189119.169357:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189119.169380:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d97d7d85 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664291328:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189119.169400:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189119.169442:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664290816 00000100:00100000:0.0:1776189119.170149:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1351 00000100:00100000:0.0:1776189119.170296:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f07038bd pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664290816:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189119.170358:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f07038bd pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664290816:12345-0@lo:400:kworker/u8:1.0 Request processed in 62us (2275us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189119.170379:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1351 00000100:00100000:0.0:1776189119.170417:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664290880 00000100:00100000:0.0:1776189119.170441:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664291328 00000100:00100000:0.0:1776189119.170460:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9745 00000100:00100000:0.0:1776189119.170464:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a93ab5a8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664290880:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189119.170495:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a93ab5a8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664290880:12345-0@lo:400:kworker/u8:1.0 Request processed in 31us (1138us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189119.170501:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9745 00000100:00100000:0.0:1776189119.170513:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9746 00000100:00100000:0.0:1776189119.170517:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c0d0294 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664291328:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189119.170541:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c0d0294 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664291328:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (1141us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189119.170547:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9746 00000100:00100000:0.0:1776189119.170589:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e5ef59b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664290880:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189119.170601:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d97d7d85 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664291328:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189119.170608:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000071c1b92f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664290816:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189119.170685:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664291264 00000100:00100000:0.0:1776189119.170715:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8401 00000100:00100000:0.0:1776189119.170719:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004d99e80b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664291264:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189119.170754:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004d99e80b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664291264:12345-0@lo:400:kworker/u8:1.0 Request processed in 34us (1475us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189119.170759:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8401 00000100:00100000:0.0:1776189119.170785:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009fddac3a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664291264:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189119.172916:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189119.172966:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f61bb34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664291072:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189119.173055:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664290944 00000100:00100000:3.0:1776189119.173088:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189119.173405:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189119.173468:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189119.173504:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046db4947 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664290752:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189119.173532:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189119.173549:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005ab67562 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664291200:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189119.173571:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189119.173592:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004eaf09e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664291456:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189119.173608:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189119.173634:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664290688 00000100:00100000:2.0:1776189119.173672:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664290752 00000100:00100000:2.0:1776189119.173687:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6586 00000100:00100000:2.0:1776189119.173697:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f326e309 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664290688:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189119.173719:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 486067136 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189119.173757:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f326e309 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664290688:12345-0@lo:13:osp-pre-0-1.0 Request processed in 63us (289us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189119.173763:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6586 00000100:00100000:2.0:1776189119.173779:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6587 00000100:00100000:2.0:1776189119.173783:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e463b423 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664290752:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189119.173983:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189119.174010:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e463b423 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664290752:12345-0@lo:13:osp-pre-1-1.0 Request processed in 226us (477us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189119.174015:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6587 00000100:00100000:2.0:1776189119.174041:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664291200 00000100:00100000:2.0:1776189119.174059:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664291456 00000100:00100000:2.0:1776189119.174069:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8402 00000100:00100000:2.0:1776189119.174072:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d9c93ad2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664291200:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189119.174095:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d9c93ad2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664291200:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (524us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189119.174099:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8402 00000100:00100000:2.0:1776189119.174106:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8403 00000100:00100000:2.0:1776189119.174109:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000229fc20a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664291456:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189119.174123:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000229fc20a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664291456:12345-0@lo:400:kworker/u8:1.0 Request processed in 14us (515us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189119.174127:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8403 00000100:00100000:2.0:1776189119.174890:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046db4947 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664290752:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189119.175008:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005ab67562 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664291200:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189119.175021:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004eaf09e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664291456:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189119.175036:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003258ba66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664290688:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189119.175096:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e071bb1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664291136:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189119.175137:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189119.175342:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664291136 00000100:00100000:2.0:1776189119.175408:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9070 00000100:00100000:2.0:1776189119.175412:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0c7825a pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664291136:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189119.175425:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189119.176017:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664291072 00000100:00100000:3.0:1776189119.176039:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57212 00000100:00100000:3.0:1776189119.176047:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001ce461df pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664290944:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189119.176054:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57213 00010000:02000000:3.0:1776189119.176059:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189119.176061:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000981bac76 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664291072:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189119.176072:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189119.176120:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001ce461df pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664290944:12345-0@lo:38:kworker/u8:1.0 Request processed in 72us (3209us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189119.176125:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000981bac76 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664291072:12345-0@lo:38:kworker/u8:1.0 Request processed in 63us (3115us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189119.176133:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57212 00000100:00100000:0.0:1776189119.176143:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57213 00000100:00080000:3.0:1776189119.176203:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189119.176208:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189119.176212:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e0967c51 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664290944:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776189119.176223:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189119.176226:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189119.176228:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f61bb34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664291072:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189119.176647:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007df90c3e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664291392:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189119.176677:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189119.176732:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664291392 00000100:00100000:1.0:1776189119.176765:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8404 00000100:00100000:1.0:1776189119.176769:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd47b990 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664291392:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189119.176778:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664291008 00000100:00100000:0.0:1776189119.176801:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9747 00000100:00100000:0.0:1776189119.176805:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006cdcc0ce pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664291008:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189119.176807:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd47b990 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664291392:12345-0@lo:400:kworker/u8:1.0 Request processed in 38us (131us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189119.176816:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8404 00000100:00100000:0.0:1776189119.176832:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006cdcc0ce pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664291008:12345-0@lo:400:kworker/u8:1.0 Request processed in 26us (3428us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189119.176838:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9747 00000100:00100000:1.0:1776189119.176844:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000051b336a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664291008:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189119.176852:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007df90c3e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664291392:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189119.199494:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0c7825a pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664291136:12345-0@lo:38:kworker/u8:1.0 Request processed in 24073us (24349us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189119.199509:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9070 00000100:00080000:3.0:1776189119.201773:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189119.201783:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189119.201789:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e071bb1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664291136:0@lo:38:kworker/u8:1.0 00010000:02020000:1.0:1776189119.420285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189119.488306:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189119.676398:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189119.932514:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189120.188391:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189120.444510:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189120.508502:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189120.700431:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189120.956473:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189121.212311:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189121.469262:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189121.532509:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189121.726094:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189121.980560:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189122.237523:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189122.492500:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189122.556475:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189122.748300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189122.938219:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189122.938330:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189122.939331:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654943616 00000100:00100000:2.0:1776189122.939391:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1352 00000100:00100000:2.0:1776189122.939397:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005039a00d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654943616:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189122.939467:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005039a00d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654943616:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 69us (1138us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189122.939475:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1352 00000100:00100000:3.0:1776189122.941002:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189122.943483:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189122.943553:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654943808 00000100:00100000:2.0:1776189122.943606:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 20352 00000100:00100000:2.0:1776189122.943612:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000020eb6bf9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+5:2277:x1862459654943808:12345-192.168.203.25@tcp:9:kworker/u8:3.0 00000020:00000080:2.0:1776189122.943648:0:186995:0:(genops.c:1442:class_disconnect()) disconnect: cookie 0xed5c52a46e435ff2 00000100:00080000:2.0:1776189122.943665:0:186995:0:(import.c:86:import_set_state_nolock()) 000000003b0fe0b8 : changing import state from FULL to CLOSED 00000001:00000004:2.0:1776189122.943706:0:186995:0:(tgt_lastrcvd.c:705:tgt_server_data_update()) lustre-OST0000_UUID: mount_count is 7, last_transno is 30064775028 00000100:00100000:2.0:1776189122.943922:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000020eb6bf9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+2:2277:x1862459654943808:12345-192.168.203.25@tcp:9:kworker/u8:3.0 Request processed in 309us (445us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189122.943942:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 20352 00010000:00010000:2.0:1776189122.943982:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776189122.944125:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654943680 00000100:00100000:1.0:1776189122.944170:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654943744 00000100:00100000:1.0:1776189122.944207:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9748 00000100:00100000:1.0:1776189122.944212:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ad500dff pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654943680:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189122.944252:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ad500dff pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654943680:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 38us (6040us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189122.944259:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9748 00000100:00100000:1.0:1776189122.944276:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57214 00000100:00100000:1.0:1776189122.944280:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000408e2292 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654943744:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776189122.944304:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189122.944492:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000408e2292 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654943744:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 210us (3496us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189122.944501:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57214 00010000:02020000:1.0:1776189123.006733:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189123.260450:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000266e2268 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664291520:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189123.260522:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189123.260624:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664291520 00000100:00100000:2.0:1776189123.260694:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6588 00000100:00100000:2.0:1776189123.260700:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004eaf09e3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664291520:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189123.260729:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189123.260792:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004eaf09e3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664291520:12345-0@lo:13:osp-pre-0-0.0 Request processed in 91us (270us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189123.260799:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6588 00000100:00100000:3.0:1776189123.260974:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000266e2268 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664291520:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189123.264790:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776189123.264815:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e5080bed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664291584:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189123.264881:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189123.265200:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664291584 00000100:00100000:2.0:1776189123.265239:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6589 00000100:00100000:2.0:1776189123.265244:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007df90c3e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664291584:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189123.265273:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189123.265315:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007df90c3e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664291584:12345-0@lo:13:osp-pre-1-0.0 Request processed in 70us (435us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189123.265322:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6589 00000100:00100000:0.0:1776189123.269575:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e5080bed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664291584:0@lo:13:osp-pre-1-0.0 00010000:02020000:3.0:1776189123.269909:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189123.526337:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189123.580413:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189123.780293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189124.036290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:0.0:1776189124.285143:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189124.285151:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189124.285154:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189124.285163:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9266 00000100:00080000:0.0:1776189124.285167:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00080000:0.0:1776189124.285202:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189124.285223:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000051b336a8 x1862459664291776/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189124.285256:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189124.285259:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189124.285260:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189124.285264:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9266 00000100:00080000:0.0:1776189124.285266:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00080000:0.0:1776189124.285269:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189124.285278:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e0967c51 x1862459664291904/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189124.285287:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189124.285290:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189124.285291:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189124.285293:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9266 00000100:00080000:0.0:1776189124.285296:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 6s 00000100:00080000:0.0:1776189124.285299:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189124.285308:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005ab67562 x1862459664291968/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189124.285395:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f61bb34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664291840:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189124.285460:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189124.285488:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e5ef59b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664292224:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189124.285511:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189124.285535:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1776189124.285541:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046db4947 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664292032:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189124.285560:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189124.285570:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b54d6c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664292288:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189124.285587:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189124.285620:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664291840 00000100:00100000:0.0:1776189124.285659:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9749 00000100:00100000:0.0:1776189124.285664:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002ef57fef pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664291840:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189124.285708:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002ef57fef pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664291840:12345-0@lo:400:kworker/u8:1.0 Request processed in 44us (249us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189124.285715:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9749 00000100:00100000:0.0:1776189124.285781:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f61bb34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664291840:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189124.285886:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d97d7d85 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664291648:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189124.285919:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189124.285938:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009fddac3a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664292096:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189124.285955:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189124.285967:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1776189124.285971:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e071bb1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664291712:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189124.285985:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189124.285995:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000071c1b92f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664292160:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189124.286009:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189124.286033:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664291712 00000100:00100000:1.0:1776189124.286050:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664292160 00000100:00100000:1.0:1776189124.286065:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9750 00000100:00100000:1.0:1776189124.286068:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001e4997ad pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664291712:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189124.286093:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001e4997ad pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664291712:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (107us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189124.286099:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9750 00000100:00100000:1.0:1776189124.286111:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9751 00000100:00100000:1.0:1776189124.286114:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fc3b4133 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664292160:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189124.286132:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fc3b4133 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664292160:12345-0@lo:400:kworker/u8:1.0 Request processed in 17us (122us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189124.286137:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9751 00000100:00100000:1.0:1776189124.286161:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e071bb1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664291712:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189124.286168:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000071c1b92f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664292160:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189124.286413:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000051b336a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664291776:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189124.286440:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189124.286462:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e0967c51 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664291904:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189124.286480:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189124.286489:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005ab67562 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664291968:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189124.286508:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189124.286534:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664292224 00000100:00100000:3.0:1776189124.286556:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664292032 00000100:00100000:3.0:1776189124.286567:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664292288 00000100:00100000:3.0:1776189124.286581:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664292096 00000100:00100000:3.0:1776189124.286594:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8405 00000100:00100000:3.0:1776189124.286597:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000549bc579 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664292224:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189124.286632:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000549bc579 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664292224:12345-0@lo:400:kworker/u8:1.0 Request processed in 34us (1121us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189124.286637:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8405 00000100:00100000:3.0:1776189124.286649:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8406 00000100:00100000:3.0:1776189124.286651:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000154c017b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664292032:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189124.286672:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000154c017b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664292032:12345-0@lo:400:kworker/u8:1.0 Request processed in 20us (1113us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189124.286677:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8406 00000100:00100000:3.0:1776189124.286685:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8407 00000100:00100000:3.0:1776189124.286687:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000058c450e1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664292288:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189124.286708:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000058c450e1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664292288:12345-0@lo:400:kworker/u8:1.0 Request processed in 19us (1121us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189124.286712:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8407 00000100:00100000:3.0:1776189124.286720:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8408 00000100:00100000:3.0:1776189124.286737:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000611c527e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664292096:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189124.286779:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000611c527e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664292096:12345-0@lo:400:kworker/u8:1.0 Request processed in 54us (823us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189124.286784:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8408 00000100:00100000:3.0:1776189124.286819:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046db4947 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664292032:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189124.286829:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b54d6c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664292288:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189124.286837:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e5ef59b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664292224:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189124.287395:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664291776 00000100:00100000:1.0:1776189124.287431:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664291904 00000100:00100000:1.0:1776189124.287448:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57215 00000100:00100000:1.0:1776189124.287451:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000266e2268 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664291776:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189124.287465:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189124.287508:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000266e2268 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664291776:12345-0@lo:38:kworker/u8:1.0 Request processed in 56us (1068us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189124.287514:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57215 00000100:00100000:1.0:1776189124.287526:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57216 00000100:00100000:1.0:1776189124.287529:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ee8b6459 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664291904:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189124.287536:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189124.287557:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ee8b6459 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664291904:12345-0@lo:38:kworker/u8:1.0 Request processed in 27us (1076us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189124.287563:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57216 00000100:00080000:1.0:1776189124.287590:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189124.287593:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189124.287597:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000051b336a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664291776:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776189124.287605:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189124.287608:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189124.287610:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e0967c51 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664291904:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189124.287643:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009fddac3a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664292096:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189124.289368:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664291968 00000100:00100000:2.0:1776189124.289415:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9071 00000100:00100000:2.0:1776189124.289421:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dabf4d62 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664291968:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189124.289436:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189124.289496:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dabf4d62 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664291968:12345-0@lo:38:kworker/u8:1.0 Request processed in 75us (2987us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189124.289502:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9071 00000100:00080000:2.0:1776189124.289547:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189124.289550:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189124.289554:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005ab67562 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664291968:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189124.289937:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776189124.289949:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094a1a2a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664292352:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189124.289985:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189124.290531:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664292352 00000100:00100000:1.0:1776189124.290559:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6590 00000100:00100000:1.0:1776189124.290563:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000c494ab9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664292352:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189124.290581:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189124.290615:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000c494ab9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664292352:12345-0@lo:13:osp-pre-0-1.0 Request processed in 51us (631us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189124.290620:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6590 00000100:00100000:0.0:1776189124.290689:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664291648 00000100:00100000:0.0:1776189124.290719:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1353 00000100:00100000:0.0:1776189124.290722:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000080a5522a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664291648:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189124.290990:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c9476df4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664292416:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189124.291019:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189124.291057:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094a1a2a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664292352:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189124.291339:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000080a5522a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664291648:12345-0@lo:400:kworker/u8:1.0 Request processed in 615us (5419us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189124.291349:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1353 00000100:00100000:3.0:1776189124.291910:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664292416 00000100:00100000:3.0:1776189124.291954:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6591 00000100:00100000:3.0:1776189124.291959:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007fe5f85f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664292416:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189124.291974:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189124.292024:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007fe5f85f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664292416:12345-0@lo:13:osp-pre-1-1.0 Request processed in 64us (1004us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189124.292031:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6591 00000100:00100000:2.0:1776189124.292148:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d97d7d85 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664291648:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189124.292843:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c9476df4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664292416:0@lo:13:osp-pre-1-1.0 00010000:02020000:3.0:1776189124.293201:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189124.551208:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189124.604463:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189124.804293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189125.060292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189125.316289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189125.572763:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189125.629008:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189125.828306:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189126.084817:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189126.342485:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189126.596432:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189126.653050:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189126.852366:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189127.108793:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189127.364299:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189127.622336:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189127.676467:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189127.876836:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189128.060052:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189128.060112:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189128.060150:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189128.060217:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654943936 00000100:00100000:0.0:1776189128.060260:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654944000 00000100:00100000:3.0:1776189128.060281:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9752 00000100:00100000:3.0:1776189128.060288:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005c66c1fe pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654943936:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189128.060297:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57217 00000100:00100000:0.0:1776189128.060303:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d7babd95 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654944000:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776189128.060319:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189128.060390:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d7babd95 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654944000:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 86us (239us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189128.060398:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57217 00000100:00100000:3.0:1776189128.060855:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005c66c1fe pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654943936:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 564us (809us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189128.060864:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9752 00000100:00100000:1.0:1776189128.060871:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654943872 00000100:00100000:1.0:1776189128.060912:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1354 00000100:00100000:1.0:1776189128.060918:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f99b7a85 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654943872:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189128.060949:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f99b7a85 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654943872:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 30us (837us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189128.060957:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1354 00010000:02020000:1.0:1776189128.132290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189128.380993:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776189128.381015:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e5080bed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664292480:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189128.381081:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189128.381194:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664292480 00000100:00100000:0.0:1776189128.381239:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6592 00000100:00100000:0.0:1776189128.381245:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c9476df4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664292480:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189128.381274:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189128.381334:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c9476df4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664292480:12345-0@lo:13:osp-pre-1-0.0 Request processed in 88us (253us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189128.381341:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6592 00000100:00100000:1.0:1776189128.381493:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e5080bed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664292480:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189128.382118:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e5080bed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664292544:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189128.382155:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189128.382260:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664292544 00000100:00100000:2.0:1776189128.382303:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6593 00000100:00100000:2.0:1776189128.382307:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cb00f0cd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664292544:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189128.382327:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189128.382362:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cb00f0cd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664292544:12345-0@lo:13:osp-pre-0-0.0 Request processed in 55us (207us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189128.382369:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6593 00000100:00100000:1.0:1776189128.382584:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e5080bed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664292544:0@lo:13:osp-pre-0-0.0 00010000:02020000:1.0:1776189128.388294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189128.644587:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189128.700369:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00100000:3.0:1776189128.898805:0:170829:0:(tgt_lastrcvd.c:541:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f 00000020:00000080:3.0:1776189128.898813:0:170829:0:(genops.c:985:class_export_put()) final put 000000008920aa8e/ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f 00000020:00000080:2.0:1776189128.898909:0:134480:0:(genops.c:944:class_export_destroy()) destroying export 000000008920aa8e/ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f for lustre-OST0000 00000020:00000080:2.0:1776189128.898969:0:134480:0:(genops.c:1164:obd_zombie_import_free()) destroying import 000000003b0fe0b8 for lustre-OST0000 00010000:02020000:1.0:1776189128.900629:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189129.157203:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189129.404426:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776189129.404450:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cae4b12b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664292608:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189129.404520:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189129.404609:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664292608 00000100:00100000:1.0:1776189129.404671:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6594 00000100:00100000:1.0:1776189129.404677:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ed846a61 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664292608:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189129.404698:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189129.404749:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ed846a61 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664292608:12345-0@lo:13:osp-pre-1-1.0 Request processed in 70us (230us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189129.404756:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6594 00000100:00100000:0.0:1776189129.404938:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000022e717c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664292672:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189129.404967:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189129.405017:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cae4b12b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664292608:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189129.405083:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664292672 00000100:00100000:3.0:1776189129.405116:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6595 00000100:00100000:3.0:1776189129.405120:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bd5316a5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664292672:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776189129.405131:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189129.405161:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bd5316a5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664292672:12345-0@lo:13:osp-pre-0-1.0 Request processed in 39us (193us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189129.405167:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6595 00000100:00080000:1.0:1776189129.405286:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189129.405291:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189129.405295:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189129.405299:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9271 00000100:00080000:1.0:1776189129.405302:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776189129.405305:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189129.405324:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000071a19d57 x1862459664292864/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189129.405345:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189129.405348:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189129.405350:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189129.405353:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9271 00000100:00080000:1.0:1776189129.405355:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776189129.405358:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189129.405368:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c5a99ce9 x1862459664292992/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189129.405376:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189129.405378:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189129.405379:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189129.405382:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9271 00000100:00080000:1.0:1776189129.405383:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776189129.405386:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189129.405395:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000098305fef x1862459664293056/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189129.405429:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b75a9f7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664292800:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189129.405460:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189129.405480:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cae4b12b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664293248:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189129.405500:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189129.405514:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1776189129.405519:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e72d32a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664292736:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189129.405544:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189129.405556:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002ea466a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664293184:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189129.405577:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189129.405601:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664292800 00000100:00100000:1.0:1776189129.405626:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664293248 00000100:00100000:1.0:1776189129.405642:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9753 00000100:00100000:1.0:1776189129.405658:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bec72d87 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664292800:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189129.405728:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bec72d87 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664292800:12345-0@lo:400:kworker/u8:1.0 Request processed in 68us (266us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189129.405734:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9753 00000100:00100000:1.0:1776189129.405746:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9754 00000100:00100000:1.0:1776189129.405749:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000086fda571 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664293248:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189129.405773:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000086fda571 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664293248:12345-0@lo:400:kworker/u8:1.0 Request processed in 23us (272us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189129.405778:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9754 00000100:00100000:1.0:1776189129.405802:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b75a9f7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664292800:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189129.405810:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cae4b12b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664293248:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189129.406774:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664292736 00000100:00100000:0.0:1776189129.406825:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1355 00000100:00100000:0.0:1776189129.406830:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000afb92d61 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664292736:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189129.406892:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000afb92d61 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664292736:12345-0@lo:400:kworker/u8:1.0 Request processed in 61us (1347us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189129.406899:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1355 00000100:00100000:0.0:1776189129.406947:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f12977ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664293120:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189129.406975:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189129.407011:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000022e717c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664292672:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189129.407028:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776189129.407033:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f4820b3d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664292928:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189129.407056:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189129.407080:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664293184 00000100:00100000:0.0:1776189129.407105:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664293120 00000100:00100000:0.0:1776189129.407122:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8409 00000100:00100000:0.0:1776189129.407126:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f9e630d1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664293184:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189129.407159:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f9e630d1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664293184:12345-0@lo:400:kworker/u8:1.0 Request processed in 33us (1582us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189129.407165:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8409 00000100:00100000:0.0:1776189129.407204:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8410 00000100:00100000:0.0:1776189129.407207:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004490682c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664293120:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189129.407238:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004490682c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664293120:12345-0@lo:400:kworker/u8:1.0 Request processed in 30us (265us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189129.407244:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8410 00000100:00100000:0.0:1776189129.407267:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664292928 00000100:00100000:0.0:1776189129.407287:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9755 00000100:00100000:0.0:1776189129.407290:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b4a5ce06 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664292928:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189129.407310:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b4a5ce06 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664292928:12345-0@lo:400:kworker/u8:1.0 Request processed in 20us (255us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189129.407313:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9755 00000100:00100000:0.0:1776189129.407329:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f4820b3d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664292928:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189129.407336:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f12977ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664293120:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189129.407401:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e72d32a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664292736:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189129.407408:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002ea466a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664293184:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189129.407548:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000071a19d57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664292864:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189129.407578:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189129.407601:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c5a99ce9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664292992:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189129.407622:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189129.407642:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000098305fef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664293056:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189129.407679:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189129.407710:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664292864 00000100:00100000:2.0:1776189129.407730:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664292992 00000100:00100000:2.0:1776189129.407744:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57218 00000100:00100000:2.0:1776189129.407747:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e5080bed pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664292864:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189129.407760:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189129.407798:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e5080bed pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664292864:12345-0@lo:38:kworker/u8:1.0 Request processed in 49us (219us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189129.407804:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57218 00000100:00100000:2.0:1776189129.407815:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57219 00000100:00100000:2.0:1776189129.407818:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d97d7d85 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664292992:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189129.407824:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189129.407845:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d97d7d85 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664292992:12345-0@lo:38:kworker/u8:1.0 Request processed in 26us (224us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189129.407850:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57219 00000100:00100000:2.0:1776189129.407870:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664293056 00000100:00100000:2.0:1776189129.407888:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9072 00000100:00100000:2.0:1776189129.407891:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003258ba66 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664293056:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189129.407898:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189129.407918:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003258ba66 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664293056:12345-0@lo:38:kworker/u8:1.0 Request processed in 26us (239us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189129.407923:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9072 00000100:00080000:2.0:1776189129.407977:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189129.407981:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189129.407985:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000071a19d57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664292864:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776189129.407993:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189129.407996:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189129.407998:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c5a99ce9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664292992:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776189129.408005:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189129.408008:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189129.408010:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000098305fef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664293056:0@lo:38:kworker/u8:1.0 00010000:02020000:1.0:1776189129.412423:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189129.668320:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189129.724758:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189129.924297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189130.180379:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189130.436656:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189130.693913:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189130.749811:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189130.948506:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189131.204352:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189131.460881:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189131.716545:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189131.773516:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189131.972442:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189132.228338:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189132.484834:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189132.740331:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189132.796393:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189132.996865:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189133.184799:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189133.184857:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189133.184898:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189133.185217:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654944064 00000100:00100000:1.0:1776189133.185289:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1356 00000100:00100000:1.0:1776189133.185295:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008f85e8f7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654944064:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1776189133.185364:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008f85e8f7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654944064:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 69us (572us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189133.185372:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1356 00000100:00100000:3.0:1776189133.186674:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654944128 00000100:00100000:3.0:1776189133.186729:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654944192 00000100:00100000:3.0:1776189133.186891:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9756 00000100:00100000:3.0:1776189133.186898:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002e9e0e35 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654944128:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1776189133.186964:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002e9e0e35 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654944128:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 66us (2107us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189133.186971:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9756 00000100:00100000:3.0:1776189133.186988:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57220 00000100:00100000:3.0:1776189133.186991:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000055462606 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654944192:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:3.0:1776189133.187006:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776189133.187031:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000055462606 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654944192:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 39us (2134us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189133.187036:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57220 00010000:02020000:1.0:1776189133.252297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189133.500532:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000045a10d8d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664293312:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189133.500622:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189133.500710:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664293312 00000100:00100000:0.0:1776189133.500763:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6596 00000100:00100000:0.0:1776189133.500768:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003afc110b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664293312:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189133.500800:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189133.501234:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cace85d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664293376:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189133.501263:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189133.501294:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664293376 00000100:00100000:1.0:1776189133.501316:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6597 00000100:00100000:1.0:1776189133.501319:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000098305fef pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664293376:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189133.501334:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189133.501360:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000098305fef pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664293376:12345-0@lo:13:osp-pre-0-0.0 Request processed in 40us (97us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189133.501367:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6597 00000100:00100000:1.0:1776189133.501410:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cace85d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664293376:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189133.504274:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003afc110b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664293312:12345-0@lo:13:osp-pre-1-0.0 Request processed in 3496us (3649us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189133.504288:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6596 00000100:00100000:3.0:1776189133.506685:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000045a10d8d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664293312:0@lo:13:osp-pre-1-0.0 00010000:02020000:1.0:1776189133.508274:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189133.764298:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189133.820545:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189134.020375:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189134.276298:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:0.0:1776189134.524483:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189134.524507:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189134.524511:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189134.524550:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9276 00000100:00080000:0.0:1776189134.524568:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:0.0:1776189134.524575:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189134.524608:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009fddac3a x1862459664293568/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189134.524822:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004be216e7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664293440:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189134.524890:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189134.524922:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c5a99ce9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664293760:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189134.524946:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189134.524986:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776189134.524991:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005ab67562 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664293504:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189134.525019:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189134.525059:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664293760 00000100:00100000:1.0:1776189134.525110:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6598 00000100:00100000:1.0:1776189134.525115:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000071a19d57 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664293760:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189134.525138:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189134.525207:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000071a19d57 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664293760:12345-0@lo:13:osp-pre-1-1.0 Request processed in 90us (260us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189134.525217:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6598 00000100:00100000:1.0:1776189134.525266:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c5a99ce9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664293760:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189134.525299:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664293440 00000100:00100000:1.0:1776189134.525333:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1357 00000100:00100000:1.0:1776189134.525337:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000034f05964 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664293440:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189134.525379:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000034f05964 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664293440:12345-0@lo:400:kworker/u8:1.0 Request processed in 41us (490us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189134.525385:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1357 00000100:00100000:1.0:1776189134.525421:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004be216e7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664293440:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189134.525446:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664293504 00000100:00100000:1.0:1776189134.525472:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9757 00000100:00100000:1.0:1776189134.525481:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002ea466a7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664293504:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189134.525515:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002ea466a7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664293504:12345-0@lo:400:kworker/u8:1.0 Request processed in 34us (496us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189134.525520:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9757 00000100:00100000:1.0:1776189134.525553:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005ab67562 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664293504:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189134.525580:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009fddac3a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664293568:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189134.525609:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189134.525636:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664293568 00000100:00100000:1.0:1776189134.525655:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57221 00000100:00100000:1.0:1776189134.525659:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c5a99ce9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664293568:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189134.525676:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189134.525704:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c5a99ce9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664293568:12345-0@lo:38:kworker/u8:1.0 Request processed in 45us (95us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189134.525710:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57221 00000100:00080000:1.0:1776189134.525734:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189134.525737:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189134.525740:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009fddac3a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664293568:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189134.525878:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e0967c51 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664293632:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189134.525908:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189134.525928:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776189134.525933:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000022e717c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664293696:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189134.525953:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189134.525985:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664293632 00000100:00100000:1.0:1776189134.526007:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9758 00000100:00100000:1.0:1776189134.526014:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e72d32a7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664293632:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189134.526049:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e72d32a7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664293632:12345-0@lo:400:kworker/u8:1.0 Request processed in 38us (141us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189134.526055:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9758 00000100:00100000:1.0:1776189134.526086:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e0967c51 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664293632:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189134.526108:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664293696 00000100:00100000:1.0:1776189134.526133:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6599 00000100:00100000:1.0:1776189134.526137:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f12977ad pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664293696:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189134.526151:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189134.526192:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f12977ad pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664293696:12345-0@lo:13:osp-pre-0-1.0 Request processed in 54us (239us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189134.526196:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6599 00000100:00100000:1.0:1776189134.526441:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000022e717c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664293696:0@lo:13:osp-pre-0-1.0 00000100:00080000:0.0:1776189134.526467:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189134.526473:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189134.526476:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189134.526481:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9276 00000100:00080000:0.0:1776189134.526484:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00080000:0.0:1776189134.526507:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189134.526722:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000051b336a8 x1862459664293824/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189134.526743:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189134.526747:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189134.526748:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189134.526751:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9276 00000100:00080000:0.0:1776189134.526753:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00080000:0.0:1776189134.526756:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189134.526780:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e5ef59b0 x1862459664293888/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189134.526895:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000051b336a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664293824:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189134.526923:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046db4947 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664293952:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189134.526932:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189134.526952:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e5ef59b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664293888:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189134.526987:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189134.526993:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189134.527022:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664293888 00000100:00100000:1.0:1776189134.527048:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9073 00000100:00100000:1.0:1776189134.527053:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cae4b12b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664293888:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189134.527053:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664293952 00010000:02000000:1.0:1776189134.527064:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189134.527096:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cae4b12b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664293888:12345-0@lo:38:kworker/u8:1.0 Request processed in 43us (111us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189134.527102:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9073 00000100:00100000:2.0:1776189134.527115:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8411 00000100:00100000:2.0:1776189134.527120:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000050c951fb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664293952:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189134.527124:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664293824 00000100:00100000:2.0:1776189134.527203:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000050c951fb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664293952:12345-0@lo:400:kworker/u8:1.0 Request processed in 83us (223us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189134.527227:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8411 00000100:00100000:1.0:1776189134.527237:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57222 00000100:00100000:1.0:1776189134.527242:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f4820b3d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664293824:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189134.527250:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189134.527277:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046db4947 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664293952:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189134.527338:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000071c1b92f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664294016:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189134.527376:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189134.527401:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664294016 00000100:00100000:2.0:1776189134.527449:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8412 00000100:00100000:2.0:1776189134.527454:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000723457d6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664294016:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189134.527503:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000723457d6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664294016:12345-0@lo:400:kworker/u8:1.0 Request processed in 48us (139us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189134.527513:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f4820b3d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664293824:12345-0@lo:38:kworker/u8:1.0 Request processed in 268us (580us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189134.527520:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57222 00000100:00080000:1.0:1776189134.527556:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189134.527559:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189134.527562:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000051b336a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664293824:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776189134.527570:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189134.527572:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189134.527574:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e5ef59b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664293888:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189134.527609:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8412 00000100:00100000:2.0:1776189134.527742:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000071c1b92f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664294016:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189134.528065:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776189134.528264:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e071bb1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664294080:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189134.528309:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189134.528446:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664294080 00000100:00100000:1.0:1776189134.528903:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9759 00000100:00100000:1.0:1776189134.528908:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000020ebaeac pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664294080:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189134.529010:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000020ebaeac pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664294080:12345-0@lo:400:kworker/u8:1.0 Request processed in 100us (700us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189134.529016:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9759 00000100:00100000:3.0:1776189134.529159:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e071bb1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664294080:0@lo:400:kworker/u8:1.0 00010000:02020000:1.0:1776189134.532358:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189134.788552:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189134.848230:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189135.044813:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189135.300276:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189135.556436:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189135.812313:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189135.870030:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189136.073645:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189136.324969:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189136.580303:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189136.836873:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189136.892497:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189137.093833:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189137.348297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189137.604698:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189137.860559:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189137.916483:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189138.116453:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189138.301075:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189138.301222:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654944256 00000100:00100000:0.0:1776189138.301294:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1358 00000100:00100000:0.0:1776189138.301301:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d6b85e7e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654944256:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189138.301378:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d6b85e7e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654944256:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 76us (309us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189138.301385:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1358 00000100:00100000:2.0:1776189138.309627:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189138.309762:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189138.310814:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654944320 00000100:00100000:1.0:1776189138.310877:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654944384 00000100:00100000:1.0:1776189138.310912:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9760 00000100:00100000:1.0:1776189138.310922:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c68f859e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654944320:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189138.311344:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c68f859e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654944320:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 421us (1724us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189138.311403:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57223 00000100:00100000:3.0:1776189138.311409:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000037fc1275 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654944384:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:3.0:1776189138.311425:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189138.311442:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9760 00000100:00100000:3.0:1776189138.311472:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000037fc1275 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654944384:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 54us (1702us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189138.311479:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57223 00010000:02020000:3.0:1776189138.372296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189138.620738:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005ecd069c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664294144:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189138.620810:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189138.620852:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776189138.620858:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b75a9f7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664294208:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189138.620881:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189138.620882:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664294144 00000100:00100000:1.0:1776189138.620927:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664294208 00000100:00100000:3.0:1776189138.620934:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6600 00000100:00100000:3.0:1776189138.620939:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000094a1a2a8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664294144:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189138.620945:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6601 00000100:00100000:1.0:1776189138.620948:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022e717c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664294208:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189138.620974:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00002000:00000024:3.0:1776189138.620977:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189138.621012:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022e717c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664294208:12345-0@lo:13:osp-pre-1-0.0 Request processed in 63us (131us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189138.621012:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000094a1a2a8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664294144:12345-0@lo:13:osp-pre-0-0.0 Request processed in 72us (202us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189138.621019:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6600 00000100:00100000:3.0:1776189138.621067:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b75a9f7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664294208:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189138.621080:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005ecd069c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664294144:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189138.624241:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6601 00010000:02020000:3.0:1776189138.628376:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189138.885951:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189138.940466:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189139.140283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189139.396548:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189139.645033:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776189139.645056:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b75a9f7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664294272:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189139.645123:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189139.645203:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664294272 00000100:00100000:1.0:1776189139.645246:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6602 00000100:00100000:1.0:1776189139.645251:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004be216e7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664294272:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189139.645273:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189139.645336:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004be216e7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664294272:12345-0@lo:13:osp-pre-1-1.0 Request processed in 83us (213us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189139.645342:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6602 00000100:00100000:1.0:1776189139.645387:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b75a9f7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664294272:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189139.645866:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776189139.645877:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000045a10d8d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664294336:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189139.645907:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189139.645979:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664294336 00000100:00100000:0.0:1776189139.646016:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6603 00000100:00100000:0.0:1776189139.646021:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008b54d6c4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664294336:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776189139.646034:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189139.646069:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008b54d6c4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664294336:12345-0@lo:13:osp-pre-0-1.0 Request processed in 47us (162us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189139.646075:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6603 00000100:00100000:3.0:1776189139.646219:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000045a10d8d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664294336:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189139.646526:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776189139.646592:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776189139.646602:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cab21754 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664294464:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189139.646633:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189139.646859:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664294464 00000100:00100000:1.0:1776189139.646900:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9761 00000100:00100000:1.0:1776189139.646904:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000045a10d8d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664294464:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189139.646948:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000045a10d8d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664294464:12345-0@lo:400:kworker/u8:0.0 Request processed in 42us (315us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189139.646955:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9761 00000100:00100000:3.0:1776189139.647814:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cab21754 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664294464:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189139.649683:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000adb1e111 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664294400:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189139.649763:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1776189139.649944:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189139.649949:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189139.649952:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189139.649957:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9281 00000100:00080000:2.0:1776189139.649960:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776189139.649967:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189139.650000:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000012b5e917 x1862459664294528/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189139.650040:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189139.650043:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189139.650045:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189139.650048:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9281 00000100:00080000:2.0:1776189139.650050:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1776189139.650136:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012b5e917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664294528:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189139.650168:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189139.650219:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776189139.650226:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028933edf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664294592:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189139.650251:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189139.650285:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664294528 00000100:00100000:0.0:1776189139.650336:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664294592 00000100:00100000:0.0:1776189139.650363:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9762 00000100:00100000:0.0:1776189139.650368:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e071bb1b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664294592:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189139.650414:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e071bb1b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664294592:12345-0@lo:400:kworker/u8:0.0 Request processed in 45us (164us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189139.650434:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9762 00000100:00100000:0.0:1776189139.650451:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57224 00000100:00100000:0.0:1776189139.650454:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000f61bb34 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664294528:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189139.650763:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189139.650802:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000f61bb34 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664294528:12345-0@lo:38:kworker/u8:0.0 Request processed in 346us (633us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189139.650808:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57224 00000100:00080000:0.0:1776189139.650840:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189139.650844:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189139.650847:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012b5e917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664294528:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189139.650874:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028933edf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664294592:0@lo:400:kworker/u8:0.0 00000100:00080000:2.0:1776189139.651017:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189139.651035:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a680c874 x1862459664294656/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189139.651054:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189139.651057:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189139.651058:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189139.651061:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9281 00000100:00080000:2.0:1776189139.651063:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776189139.651066:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189139.651074:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000073262ad2 x1862459664294720/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189139.651132:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cab21754 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664294912:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189139.651160:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189139.651190:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776189139.651195:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012b5e917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664294848:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189139.651220:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189139.651247:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664294848 00000100:00100000:2.0:1776189139.651275:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8413 00000100:00100000:2.0:1776189139.651279:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009cace85d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664294848:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189139.651692:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009cace85d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664294848:12345-0@lo:400:kworker/u8:0.0 Request processed in 411us (471us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189139.651699:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8413 00000100:00100000:2.0:1776189139.651736:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012b5e917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664294848:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189139.651962:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664294912 00000100:00100000:3.0:1776189139.652001:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9763 00000100:00100000:3.0:1776189139.652005:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005ecd069c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664294912:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189139.652045:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005ecd069c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664294912:12345-0@lo:400:kworker/u8:0.0 Request processed in 39us (885us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189139.652050:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9763 00000100:00100000:3.0:1776189139.652068:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776189139.652076:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028933edf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664294784:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189139.652103:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189139.652128:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664294784 00000100:00100000:3.0:1776189139.652147:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8414 00000100:00100000:3.0:1776189139.652151:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001f927d70 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664294784:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189139.652218:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001f927d70 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664294784:12345-0@lo:400:kworker/u8:0.0 Request processed in 65us (114us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189139.652223:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8414 00010000:02020000:3.0:1776189139.652244:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189139.652273:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028933edf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664294784:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189139.652800:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a680c874 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664294656:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189139.652839:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189139.652866:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000073262ad2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664294720:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189139.652891:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189139.652925:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664294720 00000100:00100000:0.0:1776189139.652950:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9074 00000100:00100000:0.0:1776189139.652954:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e5ef59b0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664294720:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189139.652968:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189139.653000:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e5ef59b0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664294720:12345-0@lo:38:kworker/u8:0.0 Request processed in 45us (109us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189139.653006:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9074 00000100:00080000:0.0:1776189139.653034:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189139.653038:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189139.653041:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000073262ad2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664294720:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189139.653117:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664294656 00000100:00100000:2.0:1776189139.653148:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57225 00000100:00100000:2.0:1776189139.653153:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000071c1b92f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664294656:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189139.653160:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189139.653214:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000071c1b92f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664294656:12345-0@lo:38:kworker/u8:0.0 Request processed in 60us (375us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189139.653220:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57225 00000100:00100000:2.0:1776189139.653250:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cab21754 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664294912:0@lo:400:kworker/u8:0.0 00000100:00080000:2.0:1776189139.653271:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189139.653273:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189139.653275:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a680c874 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664294656:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189139.653917:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664294400 00000100:00100000:1.0:1776189139.654001:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1359 00000100:00100000:1.0:1776189139.654006:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004c446be7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664294400:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189139.654066:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004c446be7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664294400:12345-0@lo:400:kworker/u8:0.0 Request processed in 59us (4304us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189139.654072:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1359 00000100:00100000:3.0:1776189139.654135:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000adb1e111 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664294400:0@lo:400:kworker/u8:0.0 00010000:02020000:3.0:1776189139.912288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189139.966089:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189140.165216:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189140.421217:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189140.676712:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189140.932357:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189140.990154:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189141.188483:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189141.444861:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189141.703783:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189141.956433:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189142.013228:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189142.212352:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189142.468279:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189142.724257:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189142.984459:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189143.036520:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189143.236604:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189143.418335:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189143.418694:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654944448 00000100:00100000:0.0:1776189143.418754:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1360 00000100:00100000:0.0:1776189143.418761:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fe4b4330 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654944448:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189143.418858:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fe4b4330 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654944448:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 97us (530us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189143.418866:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1360 00000100:00100000:1.0:1776189143.429964:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189143.430053:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654944576 00000100:00100000:3.0:1776189143.430103:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57226 00000100:00100000:3.0:1776189143.430109:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009b75a9f7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654944576:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:3.0:1776189143.430128:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776189143.430226:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009b75a9f7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654944576:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 115us (271us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189143.430242:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57226 00000100:00100000:3.0:1776189143.432589:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189143.432845:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654944512 00000100:00100000:0.0:1776189143.432899:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9764 00000100:00100000:0.0:1776189143.432905:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005ff20648 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654944512:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189143.432987:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005ff20648 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654944512:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 81us (410us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189143.432995:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9764 00010000:02020000:1.0:1776189143.499397:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189143.741483:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776189143.741506:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a680c874 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664294976:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189143.741573:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189143.741676:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664294976 00000100:00100000:2.0:1776189143.741725:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6604 00000100:00100000:2.0:1776189143.741730:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000051b336a8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664294976:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189143.741759:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189143.741811:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000051b336a8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664294976:12345-0@lo:13:osp-pre-0-0.0 Request processed in 80us (239us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189143.741819:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6604 00000100:00100000:3.0:1776189143.742077:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a680c874 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664294976:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189143.752579:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d63ca2bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664295040:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189143.752646:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189143.753001:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664295040 00000100:00100000:0.0:1776189143.753055:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6605 00000100:00100000:0.0:1776189143.753061:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000000115796 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664295040:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189143.753090:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189143.753141:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000000115796 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664295040:12345-0@lo:13:osp-pre-1-0.0 Request processed in 79us (495us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189143.753148:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6605 00000100:00100000:3.0:1776189143.753307:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d63ca2bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664295040:0@lo:13:osp-pre-1-0.0 00010000:02020000:1.0:1776189143.756833:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189144.012745:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189144.062199:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189144.268242:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189144.525149:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189144.764706:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000002c44196 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664295104:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189144.764776:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189144.764867:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664295104 00000100:00100000:3.0:1776189144.764918:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6606 00000100:00100000:3.0:1776189144.764924:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000046db4947 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664295104:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776189144.764946:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189144.764989:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000046db4947 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664295104:12345-0@lo:13:osp-pre-0-1.0 Request processed in 64us (213us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189144.764996:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6606 00000100:00100000:3.0:1776189144.765051:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d63ca2bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664295168:0@lo:400:kworker/u8:1.0 00000100:00080000:1.0:1776189144.765052:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189144.765057:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189144.765060:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189144.765064:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9286 00000100:00080000:1.0:1776189144.765067:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189144.765070:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189144.765079:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189144.765087:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d50ac393 x1862459664295296/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189144.765102:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00080000:1.0:1776189144.765107:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1776189144.765109:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004afd2049 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664295360:0@lo:400:kworker/u8:1.0 00000100:00080000:1.0:1776189144.765110:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189144.765112:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189144.765114:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9286 00000100:00080000:1.0:1776189144.765116:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189144.765118:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189144.765131:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000017cf884a x1862459664295424/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189144.765133:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776189144.765139:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189144.765142:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189144.765143:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189144.765145:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9286 00000100:00100000:3.0:1776189144.765146:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad3c7e22 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664295232:0@lo:400:kworker/u8:1.0 00000100:00080000:1.0:1776189144.765147:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189144.765149:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189144.765159:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009f0944ec x1862459664295488/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189144.765164:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189144.765203:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664295168 00000100:00100000:1.0:1776189144.765224:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007c850b41 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664295680:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189144.765226:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1361 00000100:00100000:3.0:1776189144.765229:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000459e616f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664295168:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189144.765251:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189144.765254:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000459e616f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664295168:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (175us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189144.765262:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1361 00000100:00100000:1.0:1776189144.765265:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776189144.765270:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000fce6caf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664295744:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189144.765286:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664295360 00000100:00100000:1.0:1776189144.765290:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189144.765306:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664295232 00000100:00100000:1.0:1776189144.765331:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d50ac393 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664295296:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189144.765331:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664295680 00000100:00100000:3.0:1776189144.765345:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9765 00000100:00100000:3.0:1776189144.765349:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000db568034 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664295360:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189144.765352:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189144.765376:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000db568034 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664295360:12345-0@lo:400:kworker/u8:1.0 Request processed in 26us (243us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189144.765381:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9765 00000100:00100000:3.0:1776189144.765391:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9766 00000100:00100000:3.0:1776189144.765394:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e009ccc0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664295232:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189144.765418:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e009ccc0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664295232:12345-0@lo:400:kworker/u8:1.0 Request processed in 23us (254us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189144.765423:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9766 00000100:00100000:3.0:1776189144.765433:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9767 00000100:00100000:3.0:1776189144.765436:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001fe1325f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664295680:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189144.765466:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001fe1325f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664295680:12345-0@lo:400:kworker/u8:1.0 Request processed in 29us (215us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189144.765471:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9767 00000100:00100000:3.0:1776189144.765497:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000061a1fd5c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664295616:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189144.765519:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189144.765547:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d63ca2bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664295168:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189144.765563:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776189144.765567:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002dce0820 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664295552:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189144.765588:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189144.765612:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664295744 00000100:00100000:3.0:1776189144.765633:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664295616 00000100:00100000:3.0:1776189144.765647:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664295552 00000100:00100000:3.0:1776189144.765662:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8415 00000100:00100000:3.0:1776189144.765665:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000048c7c2e3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664295744:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189144.765692:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000048c7c2e3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664295744:12345-0@lo:400:kworker/u8:1.0 Request processed in 26us (402us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189144.765698:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8416 00000100:00100000:2.0:1776189144.765703:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006a7d5f1c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664295616:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189144.765744:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006a7d5f1c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664295616:12345-0@lo:400:kworker/u8:1.0 Request processed in 41us (225us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189144.765781:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017cf884a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664295424:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189144.765803:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189144.765816:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009f0944ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664295488:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189144.765836:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189144.779804:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8415 00000100:00100000:3.0:1776189144.779851:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8417 00000100:00100000:3.0:1776189144.779859:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007ec15c7b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664295552:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189144.779946:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007ec15c7b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664295552:12345-0@lo:400:kworker/u8:1.0 Request processed in 88us (14357us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189144.779952:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8417 00000100:00100000:3.0:1776189144.780008:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004afd2049 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664295360:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189144.780020:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad3c7e22 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664295232:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189144.780154:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664295296 00000100:00100000:3.0:1776189144.780215:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664295424 00000100:00100000:3.0:1776189144.780227:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57227 00000100:00100000:3.0:1776189144.780231:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c8b239a6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664295296:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189144.780247:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189144.780292:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c8b239a6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664295296:12345-0@lo:38:kworker/u8:1.0 Request processed in 60us (14939us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189144.780299:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57227 00000100:00100000:3.0:1776189144.780310:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57228 00000100:00100000:3.0:1776189144.780324:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000684e24e8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664295424:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189144.780331:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189144.780355:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000684e24e8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664295424:12345-0@lo:38:kworker/u8:1.0 Request processed in 31us (14552us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189144.780360:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57228 00000100:00100000:3.0:1776189144.780415:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002dce0820 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664295552:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189144.780427:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000061a1fd5c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664295616:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189144.780441:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776189144.780449:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cab21754 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664295808:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189144.780483:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189144.780511:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664295808 00000100:00100000:3.0:1776189144.780538:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6607 00000100:00100000:3.0:1776189144.780542:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000040cb0cca pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664295808:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189144.780558:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189144.780595:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000040cb0cca pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664295808:12345-0@lo:13:osp-pre-1-1.0 Request processed in 51us (113us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189144.780601:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6607 00000100:00100000:3.0:1776189144.780632:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cab21754 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664295808:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189144.780678:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664295488 00000100:00100000:3.0:1776189144.780698:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9075 00000100:00100000:3.0:1776189144.780702:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000067c9d69b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664295488:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189144.780710:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189144.780737:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000067c9d69b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664295488:12345-0@lo:38:kworker/u8:1.0 Request processed in 34us (14901us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189144.780743:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9075 00000100:00100000:3.0:1776189144.780785:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000002c44196 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664295104:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189144.780839:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000fce6caf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664295744:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189144.780850:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007c850b41 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664295680:0@lo:400:kworker/u8:1.0 00000100:00080000:3.0:1776189144.780879:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189144.780884:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189144.780887:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d50ac393 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664295296:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776189144.780896:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189144.780898:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189144.780900:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017cf884a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664295424:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776189144.780907:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189144.780910:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189144.780912:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009f0944ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664295488:0@lo:38:kworker/u8:1.0 00010000:02020000:3.0:1776189144.783916:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189144.789253:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8416 00010000:02020000:3.0:1776189145.036328:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189145.085319:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189145.292355:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189145.550949:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189145.804434:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189146.060711:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189146.124230:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189146.318242:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189146.573053:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189146.828384:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189147.084338:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189147.132518:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189147.340924:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189147.597736:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189147.852395:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189148.109000:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189148.156472:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189148.364512:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189148.540481:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189148.540527:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189148.540603:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189148.540701:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654944640 00000100:00100000:0.0:1776189148.540749:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1362 00000100:00100000:0.0:1776189148.540754:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000de62cc6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654944640:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189148.540826:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000de62cc6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654944640:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 70us (298us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189148.540835:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1362 00000100:00100000:1.0:1776189148.545656:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654944704 00000100:00100000:1.0:1776189148.545726:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9768 00000100:00100000:1.0:1776189148.545732:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000012b5e917 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654944704:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189148.545824:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000012b5e917 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654944704:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 91us (5351us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189148.545833:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9768 00000100:00100000:1.0:1776189148.545853:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654944768 00000100:00100000:1.0:1776189148.545874:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57229 00000100:00100000:1.0:1776189148.545878:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cab21754 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654944768:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776189148.545894:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189148.545919:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cab21754 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654944768:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 40us (5318us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189148.545925:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57229 00010000:02020000:3.0:1776189148.620831:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189148.860617:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776189148.860643:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a680c874 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664295872:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189148.860718:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189148.860765:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664295872 00000100:00100000:2.0:1776189148.860808:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6608 00000100:00100000:2.0:1776189148.860814:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000adb1e111 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664295872:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189148.860843:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189148.860882:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000adb1e111 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664295872:12345-0@lo:13:osp-pre-0-0.0 Request processed in 67us (164us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189148.860889:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6608 00000100:00100000:3.0:1776189148.860910:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a680c874 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664295872:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189148.861553:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776189148.861566:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f059006a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664295936:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189148.861598:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189148.863993:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664295936 00000100:00100000:0.0:1776189148.864068:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6609 00000100:00100000:0.0:1776189148.864073:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000073262ad2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664295936:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189148.864114:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189148.864213:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000073262ad2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664295936:12345-0@lo:13:osp-pre-1-0.0 Request processed in 138us (2614us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189148.864220:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6609 00000100:00100000:2.0:1776189148.864402:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f059006a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664295936:0@lo:13:osp-pre-1-0.0 00010000:02020000:3.0:1776189148.876565:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189149.132604:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189149.185070:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189149.388280:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189149.644339:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:0.0:1776189149.884495:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189149.884503:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189149.884506:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189149.884593:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9291 00000100:00080000:0.0:1776189149.884597:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776189149.884603:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189149.884720:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005ab67562 x1862459664296128/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189149.884778:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189149.884782:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189149.884784:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1776189149.884785:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00080000:0.0:1776189149.884789:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9291 00000100:00080000:0.0:1776189149.884793:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776189149.884796:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189149.884808:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005c6d79ae x1862459664296256/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189149.884808:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e0967c51 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664296000:0@lo:400:kworker/u8:1.0 00000100:00080000:0.0:1776189149.884820:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189149.884824:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189149.884825:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189149.884828:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9291 00000100:00080000:0.0:1776189149.884842:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776189149.884846:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189149.884857:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e7c2ffe2 x1862459664296320/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189149.884905:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005cd76944 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664296192:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189149.884965:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189149.885004:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007f770c5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664296576:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189149.885029:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189149.885051:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1776189149.885057:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009fddac3a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664296064:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189149.885079:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189149.885096:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002f5aeefd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664296512:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189149.885115:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189149.885150:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664296192 00000100:00100000:2.0:1776189149.885423:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005cdf5f34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664296448:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189149.885451:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189149.885467:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028933edf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664296704:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189149.885502:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189149.885521:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1776189149.885526:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d5cfaed8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664296384:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189149.885547:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189149.885561:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000502b3e4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664296640:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189149.885578:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189149.885661:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664296064 00000100:00100000:0.0:1776189149.885883:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664296512 00000100:00100000:2.0:1776189149.885927:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664296704 00000100:00100000:3.0:1776189149.885937:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189149.885960:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6610 00000100:00100000:2.0:1776189149.885966:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008242b969 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664296704:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189149.885998:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664296576 00000100:00100000:3.0:1776189149.886022:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664296448 00000100:00100000:3.0:1776189149.886044:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8418 00000100:00100000:3.0:1776189149.886048:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c6bc9cdf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664296576:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189149.886096:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c6bc9cdf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664296576:12345-0@lo:400:kworker/u8:1.0 Request processed in 47us (1066us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189149.886103:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8418 00000100:00100000:3.0:1776189149.886116:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664296384 00000100:00100000:3.0:1776189149.886135:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664296640 00000100:00100000:1.0:1776189149.886147:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9769 00000100:00100000:1.0:1776189149.886153:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000032e99411 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664296192:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189149.886156:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8419 00002000:00000024:2.0:1776189149.886164:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189149.886165:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a680c874 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664296448:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189149.886237:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000032e99411 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664296192:12345-0@lo:400:kworker/u8:1.0 Request processed in 83us (1272us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189149.886245:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9769 00000100:00100000:2.0:1776189149.886253:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008242b969 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3362:x1862459664296704:12345-0@lo:13:osp-pre-1-1.0 Request processed in 284us (748us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189149.886263:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a680c874 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664296448:12345-0@lo:400:kworker/u8:1.0 Request processed in 90us (802us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189149.886266:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6610 00000100:00100000:1.0:1776189149.886267:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9770 00000100:00100000:1.0:1776189149.886275:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d52de5d9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664296064:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189149.886277:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8419 00000100:00100000:3.0:1776189149.886296:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8420 00000100:00100000:3.0:1776189149.886301:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b1b59934 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664296384:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189149.886312:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d52de5d9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664296064:12345-0@lo:400:kworker/u8:1.0 Request processed in 36us (1231us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189149.886319:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9770 00000100:00100000:3.0:1776189149.886347:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b1b59934 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664296384:12345-0@lo:400:kworker/u8:1.0 Request processed in 44us (799us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189149.886348:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000066892405 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664296768:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189149.886360:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8420 00000100:00100000:3.0:1776189149.886373:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8421 00000100:00100000:3.0:1776189149.886377:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e3bf25aa pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664296640:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189149.886385:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189149.886411:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e3bf25aa pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664296640:12345-0@lo:400:kworker/u8:1.0 Request processed in 32us (831us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189149.886418:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8421 00000100:00100000:1.0:1776189149.886422:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664296000 00000100:00100000:1.0:1776189149.886454:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1363 00000100:00100000:1.0:1776189149.886458:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000082b19b4e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664296000:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189149.886516:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d5cfaed8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664296384:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189149.886907:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664296768 00000100:00100000:0.0:1776189149.886936:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9771 00000100:00100000:2.0:1776189149.886939:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6611 00000100:00100000:0.0:1776189149.886945:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005be3c9f3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664296512:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189149.886997:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000082b19b4e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664296000:12345-0@lo:400:kworker/u8:1.0 Request processed in 537us (1061us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189149.887005:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1363 00000100:00100000:1.0:1776189149.887046:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e0967c51 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664296000:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189149.887100:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009fddac3a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664296064:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189149.887113:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005cd76944 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664296192:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189149.887122:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007f770c5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664296576:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189149.887158:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005ab67562 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664296128:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189149.887221:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004ae527b6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664296768:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189149.887230:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:2.0:1776189149.887239:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189149.887249:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005c6d79ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664296256:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189149.887284:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004ae527b6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664296768:12345-0@lo:13:osp-pre-0-1.0 Request processed in 337us (900us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189149.887286:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189149.887294:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6611 00000100:00100000:1.0:1776189149.887310:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e7c2ffe2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664296320:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189149.887337:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189149.887338:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664296128 00000100:00100000:2.0:1776189149.887362:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664296256 00000100:00100000:1.0:1776189149.887374:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57230 00000100:00100000:2.0:1776189149.887382:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57231 00000100:00100000:1.0:1776189149.887383:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053ba7e65 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664296128:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189149.887389:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b7a82fb0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664296256:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189149.887400:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00010000:02000000:2.0:1776189149.887400:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189149.887443:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b7a82fb0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664296256:12345-0@lo:38:kworker/u8:1.0 Request processed in 55us (160us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189149.887447:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053ba7e65 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664296128:12345-0@lo:38:kworker/u8:1.0 Request processed in 65us (230us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189149.887453:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57231 00000100:00100000:1.0:1776189149.887460:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57230 00000100:00100000:2.0:1776189149.887480:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664296320 00000100:00100000:2.0:1776189149.887497:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9076 00000100:00100000:2.0:1776189149.887500:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f059006a pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664296320:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189149.887507:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189149.887530:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f059006a pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664296320:12345-0@lo:38:kworker/u8:1.0 Request processed in 28us (193us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189149.887536:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9076 00000100:00100000:2.0:1776189149.887585:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000066892405 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664296768:0@lo:13:osp-pre-0-1.0 00000100:00080000:1.0:1776189149.887968:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189149.887973:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189149.887977:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005ab67562 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664296128:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776189149.887994:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189149.887997:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189149.887999:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005c6d79ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664296256:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776189149.888006:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189149.888009:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189149.888010:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e7c2ffe2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664296320:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189149.889623:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000502b3e4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664296640:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189149.889646:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005cdf5f34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664296448:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189149.889665:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028933edf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664296704:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189149.889802:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005be3c9f3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664296512:12345-0@lo:400:kworker/u8:1.0 Request processed in 2857us (4685us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189149.889813:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9771 00000100:00100000:3.0:1776189149.891832:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002f5aeefd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664296512:0@lo:400:kworker/u8:1.0 00010000:02020000:3.0:1776189149.901635:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189150.156953:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189150.204833:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189150.415954:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189150.668311:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189150.925293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189151.180319:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189151.230469:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:3.0:1776189151.436974:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:3.0:1776189151.692764:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189151.948962:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189152.204548:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189152.254862:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189152.461274:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189152.717267:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189152.972297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189153.228412:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189153.276454:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189153.484323:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189153.660264:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189153.661378:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654944960 00000100:00100000:2.0:1776189153.661430:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57232 00000100:00100000:2.0:1776189153.661438:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d01df670 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654944960:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:2.0:1776189153.661456:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189153.661617:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d01df670 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654944960:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 178us (1361us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189153.661631:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57232 00000100:00100000:2.0:1776189153.663298:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189153.663543:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189153.663630:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654944832 00000100:00100000:1.0:1776189153.663675:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1364 00000100:00100000:1.0:1776189153.663680:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000055012753 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654944832:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1776189153.663738:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000055012753 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654944832:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 57us (196us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189153.663745:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1364 00000100:00100000:0.0:1776189153.664368:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654944896 00000100:00100000:0.0:1776189153.664405:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9772 00000100:00100000:0.0:1776189153.664410:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004bbb2131 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654944896:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776189153.664483:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004bbb2131 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654944896:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 72us (1190us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189153.664490:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9772 00010000:02020000:1.0:1776189153.740296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189153.980405:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005ab67562 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664296832:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189153.980475:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189153.980528:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776189153.980537:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a4832ec7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664296896:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189153.980558:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189153.980621:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664296832 00000100:00100000:2.0:1776189153.980666:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664296896 00000100:00100000:2.0:1776189153.980691:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6612 00000100:00100000:2.0:1776189153.980696:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000013c0eaff pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664296832:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189153.980726:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189153.980778:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000013c0eaff pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664296832:12345-0@lo:13:osp-pre-1-0.0 Request processed in 81us (303us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189153.980784:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6612 00000100:00100000:2.0:1776189153.980800:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6613 00000100:00100000:2.0:1776189153.980803:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000051b95bd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664296896:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189153.980816:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189153.980841:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000051b95bd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664296896:12345-0@lo:13:osp-pre-0-0.0 Request processed in 37us (283us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189153.980846:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6613 00000100:00100000:2.0:1776189153.980884:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a4832ec7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664296896:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189153.980910:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005ab67562 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664296832:0@lo:13:osp-pre-1-0.0 00010000:02020000:1.0:1776189153.996286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189154.252278:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189154.306576:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189154.508265:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189154.767214:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189155.005533:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a4832ec7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664296960:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189155.005594:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189155.005735:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009fddac3a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664297024:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189155.005768:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189155.005824:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664296960 00000100:00100000:0.0:1776189155.005865:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6614 00000100:00100000:0.0:1776189155.005870:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d8f74173 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664296960:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189155.005891:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189155.005929:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d8f74173 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664296960:12345-0@lo:13:osp-pre-1-1.0 Request processed in 58us (335us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189155.005935:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6614 00000100:00100000:0.0:1776189155.006059:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664297024 00000100:00100000:0.0:1776189155.006082:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1365 00000100:00100000:0.0:1776189155.006085:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bffe4051 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664297024:12345-0@lo:400:kworker/u8:0.0 00000100:00080000:1.0:1776189155.006148:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189155.006153:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189155.006156:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189155.006160:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9296 00000100:00080000:1.0:1776189155.006193:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00080000:1.0:1776189155.006197:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189155.006213:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005ab67562 x1862459664297216/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189155.006235:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189155.006238:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189155.006240:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189155.006242:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9296 00000100:00080000:1.0:1776189155.006244:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00080000:1.0:1776189155.006247:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189155.006255:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000002c44196 x1862459664297344/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189155.006261:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189155.006263:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189155.006265:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189155.006267:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9296 00000100:00080000:1.0:1776189155.006269:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 6s 00000100:00080000:1.0:1776189155.006272:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189155.006280:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000fce6caf x1862459664297408/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189155.006323:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007c850b41 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664297472:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189155.006349:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189155.008074:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017cf884a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664297280:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189155.008144:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189155.008214:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a4832ec7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664296960:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189155.008237:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776189155.008243:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e0967c51 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664297152:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189155.008267:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189155.008310:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664297472 00000100:00100000:3.0:1776189155.008349:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8422 00000100:00100000:3.0:1776189155.008354:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000eccbf45c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664297472:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189155.008397:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000eccbf45c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664297472:12345-0@lo:400:kworker/u8:0.0 Request processed in 42us (2048us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189155.008405:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8422 00000100:00100000:3.0:1776189155.008452:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664297280 00000100:00100000:3.0:1776189155.008475:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664297152 00000100:00100000:3.0:1776189155.008493:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9773 00000100:00100000:3.0:1776189155.008496:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ae1dab54 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664297280:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189155.008530:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ae1dab54 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664297280:12345-0@lo:400:kworker/u8:0.0 Request processed in 33us (387us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189155.008536:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9773 00000100:00100000:3.0:1776189155.008548:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9774 00000100:00100000:3.0:1776189155.008551:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000066892405 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664297152:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189155.008764:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000066892405 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664297152:12345-0@lo:400:kworker/u8:0.0 Request processed in 211us (496us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189155.008770:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9774 00000100:00100000:3.0:1776189155.008792:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000032f89d8f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664297088:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189155.008817:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189155.008897:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e0967c51 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664297152:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189155.008907:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017cf884a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664297280:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189155.008960:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005ab67562 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664297216:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189155.008984:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189155.008997:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000002c44196 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664297344:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189155.009017:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189155.009028:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000fce6caf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664297408:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189155.009054:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189155.009085:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664297216 00000100:00100000:3.0:1776189155.009263:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664297344 00000100:00100000:3.0:1776189155.009279:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57233 00000100:00100000:3.0:1776189155.009283:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000017cf884a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664297216:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189155.009297:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189155.009615:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000017cf884a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664297216:12345-0@lo:38:kworker/u8:0.0 Request processed in 331us (630us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189155.009623:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57233 00000100:00100000:3.0:1776189155.009636:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57234 00000100:00100000:3.0:1776189155.009639:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e0967c51 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664297344:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189155.009646:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189155.009669:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e0967c51 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664297344:12345-0@lo:38:kworker/u8:0.0 Request processed in 29us (652us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189155.009675:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57234 00000100:00080000:3.0:1776189155.009700:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189155.009704:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189155.009707:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005ab67562 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664297216:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776189155.009715:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189155.009718:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189155.009720:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000002c44196 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664297344:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189155.009757:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664297408 00000100:00100000:0.0:1776189155.010026:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bffe4051 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664297024:12345-0@lo:400:kworker/u8:0.0 Request processed in 3938us (4257us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189155.010035:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1365 00000100:00100000:0.0:1776189155.010067:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664297088 00000100:00100000:0.0:1776189155.010101:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6615 00000100:00100000:0.0:1776189155.010106:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000028933edf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664297088:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189155.010113:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9077 00000100:00100000:3.0:1776189155.010119:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009f0944ec pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664297408:12345-0@lo:38:kworker/u8:0.0 00002000:00000024:0.0:1776189155.010124:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00010000:02000000:3.0:1776189155.010129:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189155.010310:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fc8a832b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664297600:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189155.010382:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000028933edf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664297088:12345-0@lo:13:osp-pre-0-1.0 Request processed in 274us (1563us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189155.010388:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009f0944ec pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664297408:12345-0@lo:38:kworker/u8:0.0 Request processed in 266us (1332us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189155.010391:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6615 00000100:00100000:3.0:1776189155.010399:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9077 00000100:00100000:0.0:1776189155.010417:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d63ca2bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664297536:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189155.010444:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189155.010472:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000032f89d8f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664297088:0@lo:13:osp-pre-0-1.0 00000100:00080000:3.0:1776189155.010491:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189155.010495:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189155.010501:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000fce6caf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664297408:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189155.010502:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664297536 00000100:00100000:0.0:1776189155.010528:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8423 00000100:00100000:0.0:1776189155.010533:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a943cb24 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664297536:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189155.010565:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a943cb24 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664297536:12345-0@lo:400:kworker/u8:0.0 Request processed in 32us (121us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189155.010775:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8423 00000100:00100000:0.0:1776189155.010800:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d63ca2bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664297536:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189155.010864:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189155.010947:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664297600 00000100:00100000:3.0:1776189155.010981:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9775 00000100:00100000:3.0:1776189155.010984:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003e8768bb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664297600:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189155.011053:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003e8768bb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664297600:12345-0@lo:400:kworker/u8:0.0 Request processed in 68us (190us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189155.011060:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9775 00000100:00100000:2.0:1776189155.011143:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009fddac3a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664297024:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189155.011486:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007c850b41 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664297472:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189155.011705:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fc8a832b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664297600:0@lo:400:kworker/u8:0.0 00010000:02020000:1.0:1776189155.020278:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189155.277204:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189155.324452:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189155.532319:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189155.788291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189156.044288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189156.301315:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189156.348473:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189156.556288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189156.814383:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189157.068265:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189157.324402:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189157.372397:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189157.580281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189157.838578:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189158.096747:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189158.348304:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189158.396379:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189158.604522:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189158.790424:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189158.790466:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189158.790505:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189158.790590:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654945024 00000100:00100000:1.0:1776189158.790642:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1366 00000100:00100000:1.0:1776189158.790648:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002aae1640 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654945024:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1776189158.790711:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002aae1640 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654945024:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 62us (205us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189158.790718:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1366 00000100:00100000:0.0:1776189158.791233:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654945088 00000100:00100000:0.0:1776189158.791272:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654945152 00000100:00100000:0.0:1776189158.791296:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9776 00000100:00100000:0.0:1776189158.791301:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000062ec9222 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654945088:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776189158.791333:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000062ec9222 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654945088:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 32us (919us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189158.791339:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9776 00000100:00100000:0.0:1776189158.791350:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57235 00000100:00100000:0.0:1776189158.791353:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000df218c60 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654945152:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:0.0:1776189158.791366:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189158.791387:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000df218c60 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654945152:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 33us (923us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189158.791393:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57235 00010000:02020000:1.0:1776189158.860866:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189159.100595:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776189159.100617:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009ea0f4b7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664297664:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189159.100681:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189159.100722:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776189159.100730:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fc8a832b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664297728:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189159.100751:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189159.100793:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664297664 00000100:00100000:0.0:1776189159.100833:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664297728 00000100:00100000:0.0:1776189159.100851:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6616 00000100:00100000:0.0:1776189159.100857:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004447f539 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664297664:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776189159.100884:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189159.100922:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004447f539 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664297664:12345-0@lo:13:osp-pre-0-0.0 Request processed in 65us (241us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189159.100930:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6616 00000100:00100000:0.0:1776189159.100946:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6617 00000100:00100000:0.0:1776189159.100950:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e7e6067e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664297728:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189159.100966:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189159.100993:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e7e6067e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664297728:12345-0@lo:13:osp-pre-1-0.0 Request processed in 42us (242us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189159.100999:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6617 00000100:00100000:0.0:1776189159.101070:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fc8a832b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664297728:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189159.101100:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009ea0f4b7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664297664:0@lo:13:osp-pre-0-0.0 00010000:02020000:1.0:1776189159.116294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189159.372584:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189159.421788:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189159.630605:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189159.884923:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:0.0:1776189160.124506:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189160.124515:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189160.124518:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189160.124526:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9301 00000100:00080000:0.0:1776189160.124530:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00080000:0.0:1776189160.124535:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189160.124553:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cc8ba9ef x1862459664297920/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189160.124592:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189160.124595:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189160.124596:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189160.124599:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9301 00000100:00080000:0.0:1776189160.124601:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:0.0:1776189160.124604:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189160.124613:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d4382e95 x1862459664298048/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189160.124621:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189160.124624:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189160.124626:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189160.124629:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9301 00000100:00080000:0.0:1776189160.124630:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 11s 00000100:00080000:0.0:1776189160.124633:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189160.124643:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ee179fa1 x1862459664298112/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189160.124700:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000412c3b83 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664298176:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189160.124765:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189160.124789:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002f5aeefd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664298432:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189160.124810:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189160.124829:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1776189160.124833:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a7b82f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664297792:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189160.124858:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189160.124873:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a285084b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664298240:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189160.124892:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189160.124911:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dfad713d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664297856:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189160.124934:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189160.124946:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b864aa6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664298304:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189160.124963:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189160.124973:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1776189160.124977:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc4b762b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664297984:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189160.125008:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189160.125026:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000058863251 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664298368:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189160.125046:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189160.125080:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664297792 00000100:00100000:1.0:1776189160.125120:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664297856 00000100:00100000:0.0:1776189160.125128:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1367 00000100:00100000:0.0:1776189160.125135:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006f66ddc9 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664297792:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189160.125152:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664298304 00000100:00100000:1.0:1776189160.125190:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664297984 00000100:00100000:0.0:1776189160.125210:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006f66ddc9 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664297792:12345-0@lo:400:kworker/u8:1.0 Request processed in 73us (349us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189160.125216:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9777 00000100:00100000:0.0:1776189160.125222:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1367 00000100:00100000:1.0:1776189160.125235:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e7c2ffe2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664297856:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189160.125250:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc8ba9ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664297920:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189160.125276:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a7b82f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664297792:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189160.125287:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e7c2ffe2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664297856:12345-0@lo:400:kworker/u8:1.0 Request processed in 62us (351us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189160.125295:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776189160.125304:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189160.125305:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000061a1fd5c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664298496:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189160.125314:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9777 00000100:00100000:3.0:1776189160.125320:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d4382e95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664298048:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189160.125330:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664297920 00000100:00100000:0.0:1776189160.125338:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189160.125345:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9778 00000100:00100000:1.0:1776189160.125350:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005c6d79ae pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664298304:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189160.125366:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189160.125384:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664298048 00000100:00100000:0.0:1776189160.125401:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9779 00000100:00100000:0.0:1776189160.125406:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007f770c5d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664297984:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189160.125726:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57236 00000100:00100000:2.0:1776189160.125731:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002dce0820 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664297920:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189160.125745:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189160.132851:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee179fa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664298112:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189160.132926:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189160.133759:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007f770c5d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664297984:12345-0@lo:400:kworker/u8:1.0 Request processed in 8347us (8746us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189160.133765:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005c6d79ae pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664298304:12345-0@lo:400:kworker/u8:1.0 Request processed in 8412us (8798us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189160.133775:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9779 00000100:00100000:1.0:1776189160.133786:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9778 00000100:00100000:0.0:1776189160.133814:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57237 00000100:00100000:0.0:1776189160.133820:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000fce6caf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664298048:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189160.133840:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189160.133884:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000fce6caf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664298048:12345-0@lo:38:kworker/u8:1.0 Request processed in 64us (8520us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189160.133890:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57237 00000100:00100000:1.0:1776189160.133923:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664298496 00000100:00100000:1.0:1776189160.133963:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6618 00000100:00100000:1.0:1776189160.133967:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005a7b82f1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664298496:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189160.133998:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189160.134039:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005a7b82f1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664298496:12345-0@lo:13:osp-pre-0-1.0 Request processed in 70us (8702us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189160.134045:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6618 00000100:00100000:1.0:1776189160.134086:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc4b762b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664297984:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189160.134101:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dfad713d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664297856:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189160.134111:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b864aa6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664298304:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189160.134155:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000061a1fd5c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664298496:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189160.134334:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664298112 00000100:00100000:1.0:1776189160.134357:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9078 00000100:00100000:1.0:1776189160.134361:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000002c44196 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664298112:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189160.134370:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189160.134398:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000002c44196 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664298112:12345-0@lo:38:kworker/u8:1.0 Request processed in 36us (1474us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189160.134405:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9078 00000100:00100000:1.0:1776189160.134469:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fc719d16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664298560:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189160.134498:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189160.134523:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664298560 00000100:00100000:1.0:1776189160.134543:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6619 00000100:00100000:1.0:1776189160.134547:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008dfea28a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664298560:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189160.134559:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189160.134589:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008dfea28a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664298560:12345-0@lo:13:osp-pre-1-1.0 Request processed in 41us (91us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189160.134594:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6619 00000100:00100000:1.0:1776189160.134620:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fc719d16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664298560:0@lo:13:osp-pre-1-1.0 00000100:00080000:1.0:1776189160.134645:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189160.134649:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189160.134652:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d4382e95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664298048:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776189160.134660:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189160.134663:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189160.134665:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee179fa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664298112:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189160.134705:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664298176 00000100:00100000:1.0:1776189160.134724:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664298432 00000100:00100000:1.0:1776189160.134738:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664298240 00000100:00100000:1.0:1776189160.134750:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664298368 00000100:00100000:1.0:1776189160.134762:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8424 00000100:00100000:1.0:1776189160.134765:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005cdf5f34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664298176:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189160.134801:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005cdf5f34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664298176:12345-0@lo:400:kworker/u8:1.0 Request processed in 35us (10037us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189160.134807:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8424 00000100:00100000:1.0:1776189160.134817:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8425 00000100:00100000:1.0:1776189160.134820:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000502b3e4a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664298432:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189160.134938:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000502b3e4a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664298432:12345-0@lo:400:kworker/u8:1.0 Request processed in 116us (10127us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189160.134948:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8425 00000100:00100000:1.0:1776189160.134960:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8426 00000100:00100000:1.0:1776189160.134964:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d5cfaed8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664298240:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189160.134999:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002dce0820 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664297920:12345-0@lo:38:kworker/u8:1.0 Request processed in 9264us (9697us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189160.135009:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57236 00000100:00100000:1.0:1776189160.135017:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d5cfaed8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664298240:12345-0@lo:400:kworker/u8:1.0 Request processed in 51us (10123us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189160.135024:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8426 00000100:00100000:1.0:1776189160.135041:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8427 00000100:00100000:1.0:1776189160.135045:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005cd76944 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664298368:12345-0@lo:400:kworker/u8:1.0 00000100:00080000:2.0:1776189160.135045:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189160.135051:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189160.135057:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc8ba9ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664297920:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189160.135085:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005cd76944 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664298368:12345-0@lo:400:kworker/u8:1.0 Request processed in 38us (10037us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189160.135091:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8427 00000100:00100000:1.0:1776189160.135139:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000058863251 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664298368:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189160.135166:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a285084b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664298240:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189160.135199:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000412c3b83 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664298176:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189160.135210:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002f5aeefd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664298432:0@lo:400:kworker/u8:1.0 00010000:02020000:1.0:1776189160.140512:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189160.396341:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189160.444966:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189160.652252:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189160.908363:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189161.164514:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189161.420281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189161.469227:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189161.676332:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189161.932288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189162.188295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189162.444287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189162.494073:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189162.700238:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189162.957857:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189163.212281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189163.468563:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189163.517196:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189163.724638:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189163.899282:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189163.899664:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654945216 00000100:00100000:0.0:1776189163.899779:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1368 00000100:00100000:0.0:1776189163.899787:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000002b5a0b3 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654945216:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776189163.899867:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000002b5a0b3 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654945216:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 80us (594us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189163.899876:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1368 00000100:00100000:3.0:1776189163.908466:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189163.908544:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189163.908648:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654945280 00000100:00100000:0.0:1776189163.908705:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654945344 00000100:00100000:0.0:1776189163.908732:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9780 00000100:00100000:0.0:1776189163.908739:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009fddac3a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654945280:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1776189163.908763:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57238 00000100:00100000:1.0:1776189163.908769:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005ab67562 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654945344:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:1.0:1776189163.908797:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189163.908808:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009fddac3a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654945280:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 68us (349us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189163.908816:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9780 00000100:00100000:1.0:1776189163.908832:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005ab67562 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654945344:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 61us (288us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189163.908841:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57238 00010000:02020000:1.0:1776189163.980270:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189164.221760:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001b1f5bd9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664298624:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189164.221827:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189164.221907:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664298624 00000100:00100000:0.0:1776189164.221953:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6620 00000100:00100000:0.0:1776189164.221959:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d50ac393 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664298624:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776189164.221987:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189164.222106:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fc719d16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664298688:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189164.222133:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189164.222166:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664298688 00000100:00100000:1.0:1776189164.222227:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6621 00000100:00100000:1.0:1776189164.222231:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002eebf598 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664298688:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189164.222248:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189164.222279:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002eebf598 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664298688:12345-0@lo:13:osp-pre-1-0.0 Request processed in 47us (145us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189164.222287:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6621 00000100:00100000:1.0:1776189164.222504:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fc719d16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664298688:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189164.226846:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d50ac393 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664298624:12345-0@lo:13:osp-pre-0-0.0 Request processed in 4878us (5010us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189164.226861:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6620 00000100:00100000:3.0:1776189164.234597:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001b1f5bd9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664298624:0@lo:13:osp-pre-0-0.0 00010000:02020000:1.0:1776189164.236526:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189164.494131:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189164.540465:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189164.748420:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189165.006117:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189165.244477:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fc719d16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664298752:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189165.244544:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189165.244617:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664298752 00000100:00100000:3.0:1776189165.251018:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776189165.251047:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad3c7e22 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664298816:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189165.251058:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6622 00000100:00100000:2.0:1776189165.251068:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000607ae435 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664298752:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189165.251096:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189165.251125:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189165.253662:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664298816 00000100:00100000:1.0:1776189165.253726:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6623 00000100:00100000:1.0:1776189165.253731:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009187b759 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664298816:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189165.253756:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00080000:0.0:1776189165.253849:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189165.253854:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189165.253857:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189165.253862:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9307 00000100:00080000:0.0:1776189165.253865:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:0.0:1776189165.253878:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189165.253898:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a285084b x1862459664299008/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189165.254004:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189165.254011:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189165.254012:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189165.254016:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9307 00000100:00080000:0.0:1776189165.254018:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00080000:0.0:1776189165.254021:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189165.254034:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cc8ba9ef x1862459664299136/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189165.254042:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189165.254045:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189165.254046:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189165.254049:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9307 00000100:00080000:0.0:1776189165.254051:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00080000:0.0:1776189165.254054:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189165.254064:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ee179fa1 x1862459664299200/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189165.254105:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d4382e95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664299264:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189165.262845:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002f5aeefd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664298880:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189165.262854:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000607ae435 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664298752:12345-0@lo:13:osp-pre-1-1.0 Request processed in 11784us (18307us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189165.262879:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6622 00000100:00100000:2.0:1776189165.262932:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000058863251 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664299072:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189165.262939:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189165.262976:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189165.262980:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b864aa6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664299328:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189165.263014:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189165.263035:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fc719d16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664298752:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189165.263062:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1776189165.263063:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664298880 00000100:00100000:2.0:1776189165.263070:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000412c3b83 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664298944:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189165.263097:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189165.263110:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1369 00000100:00100000:3.0:1776189165.263117:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000064f5974 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664298880:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189165.263118:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dfad713d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664299392:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189165.263144:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189165.263196:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000064f5974 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664298880:12345-0@lo:400:kworker/u8:0.0 Request processed in 77us (260us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189165.263200:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664299072 00000100:00100000:3.0:1776189165.263204:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1369 00000100:00100000:2.0:1776189165.263234:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664298944 00000100:00100000:3.0:1776189165.263242:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664299328 00000100:00100000:2.0:1776189165.263256:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664299392 00000100:00100000:3.0:1776189165.263275:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8428 00000100:00100000:3.0:1776189165.263281:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004afd2049 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664299328:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189165.263285:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9781 00000100:00100000:2.0:1776189165.263292:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f00e14c6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664299072:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189165.263324:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004afd2049 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664299328:12345-0@lo:400:kworker/u8:0.0 Request processed in 43us (309us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189165.263331:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8428 00000100:00100000:3.0:1776189165.263355:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9782 00000100:00100000:3.0:1776189165.263359:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fcf58f4c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664298944:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189165.263882:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009187b759 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664298816:12345-0@lo:13:osp-pre-0-1.0 Request processed in 10148us (12757us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189165.263892:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6623 00010000:02020000:1.0:1776189165.264124:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189165.264146:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9783 00000100:00100000:1.0:1776189165.264151:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0533a16 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664299392:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189165.264215:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0533a16 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664299392:12345-0@lo:400:kworker/u8:0.0 Request processed in 64us (1071us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189165.264222:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9783 00000100:00100000:1.0:1776189165.264286:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a285084b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664299008:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189165.264314:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189165.264924:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc8ba9ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664299136:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189165.264953:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189165.264968:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee179fa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664299200:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189165.264993:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189165.265041:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664299008 00000100:00100000:1.0:1776189165.265062:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664299136 00000100:00100000:1.0:1776189165.265072:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57239 00000100:00100000:1.0:1776189165.265075:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008dfea28a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664299008:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189165.265089:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189165.265129:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008dfea28a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664299008:12345-0@lo:38:kworker/u8:0.0 Request processed in 52us (814us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189165.265135:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57239 00000100:00100000:1.0:1776189165.265147:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57240 00000100:00100000:1.0:1776189165.265150:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002eebf598 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664299136:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189165.265156:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189165.265205:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002eebf598 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664299136:12345-0@lo:38:kworker/u8:0.0 Request processed in 52us (251us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189165.265210:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57240 00000100:00080000:1.0:1776189165.265246:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189165.265250:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189165.265253:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a285084b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664299008:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776189165.265261:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189165.265263:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189165.265265:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc8ba9ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664299136:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189165.265652:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664299200 00000100:00100000:1.0:1776189165.265676:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9079 00000100:00100000:1.0:1776189165.265680:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ef95cf3e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664299200:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189165.265688:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189165.265720:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ef95cf3e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664299200:12345-0@lo:38:kworker/u8:0.0 Request processed in 39us (726us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189165.265726:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9079 00000100:00080000:1.0:1776189165.265897:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189165.265900:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189165.265903:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee179fa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664299200:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189165.265931:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dfad713d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664299392:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189165.266031:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189165.266116:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664299264 00000100:00100000:1.0:1776189165.266153:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8429 00000100:00100000:1.0:1776189165.266157:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cc4b762b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664299264:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189165.266215:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cc4b762b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664299264:12345-0@lo:400:kworker/u8:0.0 Request processed in 57us (184us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189165.266222:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8429 00000100:00100000:0.0:1776189165.266816:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d4382e95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664299264:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189165.267794:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fcf58f4c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664298944:12345-0@lo:400:kworker/u8:0.0 Request processed in 4429us (4693us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189165.267805:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9782 00000100:00100000:3.0:1776189165.267855:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002f5aeefd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664298880:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189165.267888:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b864aa6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664299328:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189165.267910:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad3c7e22 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664298816:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189165.270113:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000412c3b83 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664298944:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189165.271137:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f00e14c6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664299072:12345-0@lo:400:kworker/u8:0.0 Request processed in 7839us (8156us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189165.271151:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9781 00000100:00100000:1.0:1776189165.275808:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000058863251 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664299072:0@lo:400:kworker/u8:0.0 00010000:02020000:1.0:1776189165.516303:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189165.569503:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189165.772917:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189166.029750:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189166.284313:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189166.541063:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189166.591272:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189166.796301:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189167.053771:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189167.308306:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189167.564289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189167.612498:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189167.821203:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189168.076287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189168.332292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189168.589421:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189168.636436:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189168.844583:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189169.026361:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189169.026408:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189169.026497:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654945408 00000100:00100000:2.0:1776189169.026562:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1370 00000100:00100000:2.0:1776189169.026568:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000084b636aa pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654945408:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189169.026633:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000084b636aa pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654945408:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 64us (225us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189169.026641:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1370 00000100:00100000:0.0:1776189169.027116:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654945472 00000100:00100000:0.0:1776189169.027157:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9784 00000100:00100000:0.0:1776189169.027162:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ad3c7e22 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654945472:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189169.027217:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ad3c7e22 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654945472:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 54us (866us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189169.027224:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9784 00000100:00100000:3.0:1776189169.029874:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189169.029948:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654945536 00000100:00100000:0.0:1776189169.029997:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57241 00000100:00100000:0.0:1776189169.030013:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d50ac393 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654945536:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776189169.030032:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189169.030102:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d50ac393 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654945536:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 97us (235us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189169.030110:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57241 00010000:02020000:1.0:1776189169.102028:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189169.340544:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000debe3528 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664299456:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189169.340617:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189169.340755:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664299456 00000100:00100000:1.0:1776189169.340811:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6624 00000100:00100000:1.0:1776189169.340817:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001929a2c4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664299456:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189169.340846:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189169.340894:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001929a2c4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664299456:12345-0@lo:13:osp-pre-1-0.0 Request processed in 76us (277us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189169.340901:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6624 00000100:00100000:0.0:1776189169.340983:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000debe3528 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664299456:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189169.341662:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002c38fc42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664299520:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189169.341700:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189169.341916:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664299520 00000100:00100000:1.0:1776189169.341957:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6625 00000100:00100000:1.0:1776189169.341962:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d4382e95 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664299520:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189169.341985:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189169.342027:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d4382e95 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664299520:12345-0@lo:13:osp-pre-0-0.0 Request processed in 64us (327us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189169.342034:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6625 00000100:00100000:0.0:1776189169.342109:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002c38fc42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664299520:0@lo:13:osp-pre-0-0.0 00010000:02020000:1.0:1776189169.356447:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189169.612243:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189169.661044:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189169.868902:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189170.125549:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189170.365192:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776189170.365215:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602ee85e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664299584:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189170.365282:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189170.365514:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664299584 00000100:00100000:3.0:1776189170.365560:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6626 00000100:00100000:3.0:1776189170.365566:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001ada75b0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664299584:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776189170.365592:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189170.365634:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001ada75b0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664299584:12345-0@lo:13:osp-pre-0-1.0 Request processed in 68us (353us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189170.365932:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602ee85e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664299584:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189170.366041:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6626 00000100:00080000:1.0:1776189170.366810:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1776189170.366917:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007c231a0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664299648:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189170.366967:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189170.366992:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776189170.366998:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000611a3c66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664299712:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189170.367024:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189170.367128:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664299648 00000100:00100000:0.0:1776189170.367162:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1371 00000100:00100000:0.0:1776189170.367167:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c1f77e46 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664299648:12345-0@lo:400:kworker/u8:0.0 00000100:00080000:1.0:1776189170.367231:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189170.367234:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189170.367239:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9312 00000100:00080000:1.0:1776189170.367242:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189170.367246:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189170.367265:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f1ff984b x1862459664299776/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189170.367293:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189170.367296:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189170.367298:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189170.367300:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9312 00000100:00080000:1.0:1776189170.367303:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189170.367306:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189170.367315:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006bb8567a x1862459664299904/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189170.367327:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189170.367330:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189170.367331:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189170.367334:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9312 00000100:00080000:1.0:1776189170.367335:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189170.367337:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189170.367347:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000eeb8de0 x1862459664299968/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189170.367393:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f1ff984b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664299776:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189170.368033:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ac2fe2ac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664299840:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189170.368040:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c1f77e46 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664299648:12345-0@lo:400:kworker/u8:0.0 Request processed in 869us (1070us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189170.368073:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189170.368094:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ef812603 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664300224:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189170.368119:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189170.368138:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776189170.368144:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000058fbcadb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664300032:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189170.368168:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189170.368214:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664300224 00000100:00100000:2.0:1776189170.368245:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664300032 00000100:00100000:2.0:1776189170.368264:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8430 00000100:00100000:2.0:1776189170.368269:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ad48824b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664300224:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189170.368306:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ad48824b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664300224:12345-0@lo:400:kworker/u8:0.0 Request processed in 36us (186us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189170.368313:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8430 00000100:00100000:2.0:1776189170.368327:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8431 00000100:00100000:2.0:1776189170.368330:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ddb9f539 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664300032:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189170.368354:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ddb9f539 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664300032:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (186us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189170.368360:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8431 00000100:00100000:2.0:1776189170.368399:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000058fbcadb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664300032:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189170.368422:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ef812603 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664300224:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189170.368458:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664299712 00000100:00100000:3.0:1776189170.368470:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a9575a34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664300160:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189170.368486:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664299840 00000100:00100000:3.0:1776189170.368507:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189170.368511:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9785 00000100:00100000:2.0:1776189170.368516:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000058863251 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664299712:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189170.368532:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776189170.368541:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5008497 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664300096:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189170.368568:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000058863251 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664299712:12345-0@lo:400:kworker/u8:0.0 Request processed in 51us (1544us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189170.368579:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9785 00000100:00100000:3.0:1776189170.368584:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189170.368593:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664300160 00000100:00100000:2.0:1776189170.368621:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9786 00000100:00100000:2.0:1776189170.368627:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b51cb18e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664299840:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189170.368630:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9787 00000100:00100000:3.0:1776189170.368636:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000075d6a9fc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664300160:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189170.368746:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1371 00000100:00100000:0.0:1776189170.368793:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664300096 00000100:00100000:1.0:1776189170.369360:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189170.369384:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bb8567a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664299904:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189170.369403:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189170.369409:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000eeb8de0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664299968:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189170.369484:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189170.369571:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664299968 00000100:00100000:1.0:1776189170.369592:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9080 00000100:00100000:1.0:1776189170.369596:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008821baff pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664299968:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189170.369611:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189170.369656:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008821baff pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664299968:12345-0@lo:38:kworker/u8:0.0 Request processed in 60us (172us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189170.369662:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9080 00000100:00100000:1.0:1776189170.369679:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ff5d45b7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664300288:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189170.369701:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189170.369746:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000611a3c66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664299712:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189170.369767:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007c231a0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664299648:0@lo:400:kworker/u8:0.0 00000100:00080000:1.0:1776189170.369802:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:2.0:1776189170.369803:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b51cb18e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664299840:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (588us total) trans 0 rc 0/0 00000100:00080000:1.0:1776189170.369807:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189170.369815:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9786 00000100:00100000:1.0:1776189170.369823:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000eeb8de0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664299968:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189170.369837:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664299776 00000100:00100000:2.0:1776189170.369850:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664299904 00000100:00100000:1.0:1776189170.369865:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8432 00000100:00100000:2.0:1776189170.369867:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57242 00000100:00100000:1.0:1776189170.369874:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bd5316a5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664300096:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189170.369875:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000603f73df pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664299776:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189170.369888:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189170.370010:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57243 00000100:00100000:0.0:1776189170.370015:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006e96e3d4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664299904:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189170.370045:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189170.372017:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bd5316a5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664300096:12345-0@lo:400:kworker/u8:0.0 Request processed in 2136us (3426us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189170.372032:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8432 00000100:00100000:1.0:1776189170.373097:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664300288 00000100:00100000:1.0:1776189170.373153:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6627 00000100:00100000:1.0:1776189170.373159:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009806e292 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664300288:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189170.373206:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189170.373277:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009806e292 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664300288:12345-0@lo:13:osp-pre-1-1.0 Request processed in 117us (3576us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189170.373283:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6627 00000100:00100000:1.0:1776189170.373335:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ff5d45b7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664300288:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189170.373374:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ac2fe2ac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664299840:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189170.373419:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5008497 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664300096:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189170.376865:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006e96e3d4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664299904:12345-0@lo:38:kworker/u8:0.0 Request processed in 6844us (7457us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189170.376865:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000603f73df pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664299776:12345-0@lo:38:kworker/u8:0.0 Request processed in 6982us (7495us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189170.376869:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000075d6a9fc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664300160:12345-0@lo:400:kworker/u8:0.0 Request processed in 8232us (8363us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189170.376886:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57242 00000100:00100000:3.0:1776189170.376893:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9787 00000100:00100000:0.0:1776189170.376896:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57243 00000100:00100000:3.0:1776189170.376956:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a9575a34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664300160:0@lo:400:kworker/u8:0.0 00000100:00080000:1.0:1776189170.377075:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189170.377080:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189170.377085:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f1ff984b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664299776:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776189170.377094:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189170.377097:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189170.377099:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bb8567a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664299904:0@lo:38:kworker/u8:0.0 00010000:02020000:1.0:1776189170.381347:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189170.636285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189170.697927:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189170.892474:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189171.148383:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189171.404457:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189171.661136:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189171.708476:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189171.916417:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189172.172445:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189172.428842:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189172.684288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189172.732424:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189172.940763:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189173.196286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189173.452291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189173.708290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189173.759125:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189173.966307:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189174.141284:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189174.141401:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189174.146348:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189174.146722:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654945600 00000100:00100000:0.0:1776189174.146777:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1372 00000100:00100000:0.0:1776189174.146867:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f055c03f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654945600:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776189174.146938:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f055c03f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654945600:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 153us (597us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189174.146946:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1372 00000100:00100000:2.0:1776189174.151909:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654945728 00000100:00100000:2.0:1776189174.151972:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654945664 00000100:00100000:2.0:1776189174.152015:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9788 00000100:00100000:2.0:1776189174.152022:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000602ee85e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654945664:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1776189174.152099:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000602ee85e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654945664:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 77us (10699us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189174.152106:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9788 00000100:00100000:2.0:1776189174.152126:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57244 00000100:00100000:2.0:1776189174.152131:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004be216e7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654945728:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:2.0:1776189174.152145:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189174.152194:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004be216e7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654945728:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 62us (10920us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189174.152201:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57244 00010000:02020000:1.0:1776189174.220341:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189174.460475:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776189174.460498:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bb8567a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664300352:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189174.460579:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189174.460677:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664300352 00000100:00100000:1.0:1776189174.460729:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6628 00000100:00100000:1.0:1776189174.460734:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a829435f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664300352:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189174.460762:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189174.460802:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a829435f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664300352:12345-0@lo:13:osp-pre-1-0.0 Request processed in 67us (224us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189174.460808:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6628 00000100:00100000:3.0:1776189174.460906:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bb8567a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664300352:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189174.461785:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001065b146 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664300416:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189174.461827:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189174.461896:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664300416 00000100:00100000:1.0:1776189174.461929:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6629 00000100:00100000:1.0:1776189174.461933:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008b54d6c4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664300416:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189174.461955:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189174.462057:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008b54d6c4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664300416:12345-0@lo:13:osp-pre-0-0.0 Request processed in 123us (230us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189174.462064:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6629 00000100:00100000:3.0:1776189174.462134:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001065b146 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664300416:0@lo:13:osp-pre-0-0.0 00010000:02020000:1.0:1776189174.476313:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189174.732498:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189174.780770:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189174.988467:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189175.244292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189175.484403:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f1ff984b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664300480:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189175.484478:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189175.484541:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664300480 00000100:00100000:1.0:1776189175.484585:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6630 00000100:00100000:1.0:1776189175.484590:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a5008497 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664300480:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189175.484609:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189175.484645:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a5008497 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664300480:12345-0@lo:13:osp-pre-1-1.0 Request processed in 54us (168us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189175.484663:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6630 00000100:00100000:1.0:1776189175.484707:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f1ff984b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664300480:0@lo:13:osp-pre-1-1.0 00000100:00080000:0.0:1776189175.485066:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189175.485070:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189175.485073:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189175.485078:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9317 00000100:00080000:0.0:1776189175.485080:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1776189175.485084:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189175.485102:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002f5aeefd x1862459664300672/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189175.485122:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189175.485125:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189175.485126:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189175.485129:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9317 00000100:00080000:0.0:1776189175.485131:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1776189175.485133:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189175.485142:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ee179fa1 x1862459664300800/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189175.485150:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189175.485152:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189175.485153:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189175.485156:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9317 00000100:00080000:0.0:1776189175.485157:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1776189175.485159:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189175.485169:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cc8ba9ef x1862459664300864/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189175.485237:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b864aa6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664300608:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189175.485270:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189175.485286:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e7e6067e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664301056:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189175.485304:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189175.485401:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002f5aeefd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664300672:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189175.485425:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189175.485438:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee179fa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664300800:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189175.485457:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189175.485472:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc8ba9ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664300864:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189175.485492:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189175.485520:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664300864 00000100:00100000:1.0:1776189175.485540:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9081 00000100:00100000:1.0:1776189175.485597:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ff5d45b7 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664300864:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189175.485612:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189175.485646:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ff5d45b7 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664300864:12345-0@lo:38:kworker/u8:1.0 Request processed in 50us (154us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189175.485662:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9081 00000100:00100000:1.0:1776189175.485683:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000412c3b83 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664300544:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189175.485711:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189175.485724:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004447f539 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664300992:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189175.485744:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189175.485764:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776189175.485768:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a285084b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664300928:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189175.485919:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664300992 00000100:00100000:2.0:1776189175.485948:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8433 00000100:00100000:2.0:1776189175.485952:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000eeb8de0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664300992:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189175.486071:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664300608 00000100:00100000:3.0:1776189175.486099:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664301056 00000100:00100000:3.0:1776189175.486112:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664300672 00000100:00100000:3.0:1776189175.486122:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664300800 00000100:00100000:3.0:1776189175.486132:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9789 00000100:00100000:3.0:1776189175.486136:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d63ca2bb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664300608:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189175.486501:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1776189175.486509:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dfad713d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664300736:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189175.486922:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000eeb8de0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664300992:12345-0@lo:400:kworker/u8:1.0 Request processed in 967us (1175us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189175.486933:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8433 00000100:00100000:2.0:1776189175.486969:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9790 00000100:00100000:2.0:1776189175.486973:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000040cb0cca pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664301056:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189175.487008:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000040cb0cca pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664301056:12345-0@lo:400:kworker/u8:1.0 Request processed in 35us (1704us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189175.487015:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9790 00000100:00100000:2.0:1776189175.487030:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57245 00000100:00100000:2.0:1776189175.487033:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f1ff984b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664300672:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189175.487046:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189175.487072:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f1ff984b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664300672:12345-0@lo:38:kworker/u8:1.0 Request processed in 37us (1646us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189175.487078:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57245 00000100:00100000:2.0:1776189175.487088:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57246 00000100:00100000:2.0:1776189175.487091:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ac2fe2ac pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664300800:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189175.487099:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189175.487122:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ac2fe2ac pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664300800:12345-0@lo:38:kworker/u8:1.0 Request processed in 30us (1665us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189175.487127:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57246 00000100:00100000:2.0:1776189175.487193:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664300544 00000100:00100000:2.0:1776189175.487231:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1373 00000100:00100000:2.0:1776189175.487235:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000353f317b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664300544:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189175.487898:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000353f317b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664300544:12345-0@lo:400:kworker/u8:1.0 Request processed in 659us (2184us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189175.487907:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1373 00000100:00100000:2.0:1776189175.487946:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000002c44196 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664301184:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189175.487978:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189175.488017:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664301184 00000100:00100000:2.0:1776189175.488046:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6631 00000100:00100000:2.0:1776189175.488050:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d0281336 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664301184:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189175.488067:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189175.488100:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d0281336 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664301184:12345-0@lo:13:osp-pre-0-1.0 Request processed in 49us (121us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189175.488107:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6631 00000100:00100000:2.0:1776189175.488155:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000002c44196 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664301184:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189175.490956:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189175.491027:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000412c3b83 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664300544:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189175.491045:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004447f539 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664300992:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189175.491101:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664300928 00000100:00100000:1.0:1776189175.491147:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8434 00000100:00100000:1.0:1776189175.491152:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007c231a0e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664300928:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189175.491357:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007c231a0e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664300928:12345-0@lo:400:kworker/u8:1.0 Request processed in 203us (408us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189175.491365:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8434 00000100:00100000:1.0:1776189175.491398:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a285084b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664300928:0@lo:400:kworker/u8:1.0 00000100:00080000:2.0:1776189175.492199:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189175.492206:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189175.492211:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002f5aeefd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664300672:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776189175.492229:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189175.492231:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189175.492235:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee179fa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664300800:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776189175.492245:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189175.492249:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189175.492251:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc8ba9ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664300864:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189175.493558:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d63ca2bb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664300608:12345-0@lo:400:kworker/u8:1.0 Request processed in 7416us (8283us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189175.493574:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9789 00000100:00100000:0.0:1776189175.496471:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189175.496525:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a7b82f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664301120:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189175.496574:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189175.496689:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b864aa6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664300608:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189175.496704:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e7e6067e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664301056:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189175.496784:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664300736 00000100:00100000:3.0:1776189175.496890:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9791 00000100:00100000:3.0:1776189175.496897:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000013c0eaff pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664300736:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189175.496898:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664301120 00000100:00100000:3.0:1776189175.496970:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000013c0eaff pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664300736:12345-0@lo:400:kworker/u8:1.0 Request processed in 73us (506us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189175.496979:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9791 00000100:00100000:0.0:1776189175.497013:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dfad713d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664300736:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189175.497354:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8435 00000100:00100000:1.0:1776189175.497359:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000051b95bd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664301120:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189175.497403:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000051b95bd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664301120:12345-0@lo:400:kworker/u8:1.0 Request processed in 44us (830us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189175.497409:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8435 00000100:00100000:0.0:1776189175.497462:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a7b82f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664301120:0@lo:400:kworker/u8:1.0 00010000:02020000:1.0:1776189175.500284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189175.758506:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189175.806510:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189176.012312:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189176.268287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189176.524885:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189176.780472:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189176.828442:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189177.036604:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189177.292356:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189177.548393:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189177.804321:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189177.852382:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189178.060349:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189178.316323:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189178.572380:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189178.828287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189178.877904:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189179.084284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189179.265461:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189179.265516:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189179.265562:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189179.265578:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654945856 00000100:00100000:0.0:1776189179.265655:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654945920 00000100:00100000:0.0:1776189179.265675:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9792 00000100:00100000:0.0:1776189179.265681:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000000115796 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654945856:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189179.265739:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57247 00000100:00100000:1.0:1776189179.265746:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000002c44196 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654945920:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00000100:00100000:0.0:1776189179.265752:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000000115796 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654945856:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 70us (299us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189179.265761:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9792 00010000:02000000:1.0:1776189179.265764:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189179.265799:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000002c44196 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654945920:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 53us (238us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189179.265806:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57247 00000100:00100000:2.0:1776189179.265976:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654945792 00000100:00100000:2.0:1776189179.266017:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1374 00000100:00100000:2.0:1776189179.266023:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003c5b81e6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654945792:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189179.266067:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003c5b81e6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654945792:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 44us (551us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189179.266082:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1374 00010000:02020000:2.0:1776189179.340459:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189179.587566:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000611a3c66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664301248:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189179.587642:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189179.589911:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664301248 00000100:00100000:1.0:1776189179.589987:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6632 00000100:00100000:1.0:1776189179.589994:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e5ef59b0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664301248:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189179.590027:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189179.590112:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e5ef59b0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664301248:12345-0@lo:13:osp-pre-1-0.0 Request processed in 118us (2470us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189179.590120:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6632 00000100:00100000:0.0:1776189179.590283:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000611a3c66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664301248:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189179.590441:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776189179.590452:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000624c000a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664301312:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189179.590488:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189179.590612:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664301312 00000100:00100000:2.0:1776189179.590646:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6633 00000100:00100000:2.0:1776189179.590650:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003a715368 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664301312:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189179.590670:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189179.590720:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003a715368 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664301312:12345-0@lo:13:osp-pre-0-0.0 Request processed in 68us (232us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189179.590726:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6633 00000100:00100000:3.0:1776189179.590791:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000624c000a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664301312:0@lo:13:osp-pre-0-0.0 00010000:02020000:2.0:1776189179.596670:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189179.854168:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189179.905206:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189180.111225:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189180.367932:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189180.606253:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9138b66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664301376:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189180.606320:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189180.606380:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664301376 00000100:00100000:0.0:1776189180.606420:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6634 00000100:00100000:0.0:1776189180.606425:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000032f89d8f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664301376:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189180.606449:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189180.606474:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009257ba59 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664301440:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189180.606487:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000032f89d8f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664301376:12345-0@lo:13:osp-pre-1-1.0 Request processed in 61us (167us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189180.606493:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6634 00000100:00100000:3.0:1776189180.606500:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189180.606512:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664301440 00000100:00080000:1.0:1776189180.606595:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189180.606600:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189180.606603:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189180.606608:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9322 00000100:00080000:1.0:1776189180.606611:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776189180.606614:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189180.606646:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005355735b x1862459664301632/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189180.606667:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189180.606670:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189180.606672:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189180.606674:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9322 00000100:00080000:1.0:1776189180.606676:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776189180.606679:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189180.606689:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006bb8567a x1862459664301760/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189180.606696:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189180.606700:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189180.606701:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189180.606713:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9322 00000100:00080000:1.0:1776189180.606716:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776189180.606718:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189180.606729:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a9575a34 x1862459664301824/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189180.606775:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000611a3c66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664301696:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189180.606804:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189180.606839:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9138b66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664301376:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189180.606856:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1776189180.606861:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000917ca813 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664301568:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189180.606882:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189180.606897:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000debe3528 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664302016:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189180.606917:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189180.606941:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664301696 00000100:00100000:1.0:1776189180.606967:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664301568 00000100:00100000:1.0:1776189180.606981:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664302016 00000100:00100000:1.0:1776189180.606993:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9793 00000100:00100000:1.0:1776189180.606997:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fc719d16 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664301696:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189180.607033:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fc719d16 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664301696:12345-0@lo:400:kworker/u8:1.0 Request processed in 35us (230us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189180.607040:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9793 00000100:00100000:1.0:1776189180.607053:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9794 00000100:00100000:1.0:1776189180.607056:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e9138b66 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664301568:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189180.607085:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e9138b66 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664301568:12345-0@lo:400:kworker/u8:1.0 Request processed in 27us (202us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189180.607091:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9794 00000100:00100000:1.0:1776189180.607101:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9795 00000100:00100000:1.0:1776189180.607104:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fc8a832b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664302016:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189180.607129:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fc8a832b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664302016:12345-0@lo:400:kworker/u8:1.0 Request processed in 23us (211us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189180.607134:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9795 00000100:00100000:1.0:1776189180.607152:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c4b16b97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664301504:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189180.607198:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189180.607214:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000058fbcadb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664301952:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189180.607237:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189180.607253:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776189180.607257:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ef812603 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664301888:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189180.607275:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189180.607297:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664301952 00000100:00100000:1.0:1776189180.607316:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664301888 00000100:00100000:1.0:1776189180.607329:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8436 00000100:00100000:1.0:1776189180.607333:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005a7b82f1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664301952:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189180.607364:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005a7b82f1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664301952:12345-0@lo:400:kworker/u8:1.0 Request processed in 30us (126us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189180.607369:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8436 00000100:00100000:1.0:1776189180.607382:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8437 00000100:00100000:1.0:1776189180.607384:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dfad713d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664301888:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189180.607407:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dfad713d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664301888:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (132us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189180.607413:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8437 00000100:00100000:1.0:1776189180.607436:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000917ca813 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664301568:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189180.607446:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000debe3528 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664302016:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189180.607454:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000611a3c66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664301696:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189180.607481:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ef812603 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664301888:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189180.607489:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000058fbcadb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664301952:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189180.607602:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6635 00000100:00100000:0.0:1776189180.607611:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000051b336a8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664301440:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776189180.607626:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189180.607657:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000051b336a8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664301440:12345-0@lo:13:osp-pre-0-1.0 Request processed in 46us (1156us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189180.607662:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6635 00000100:00100000:3.0:1776189180.619855:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009257ba59 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664301440:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189180.619935:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005355735b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664301632:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189180.619996:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189180.620021:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bb8567a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664301760:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189180.620042:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189180.620056:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a9575a34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664301824:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189180.620082:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189180.620127:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664301632 00000100:00100000:3.0:1776189180.620158:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664301760 00000100:00100000:1.0:1776189180.620987:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664301824 00000100:00100000:1.0:1776189180.621022:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9082 00000100:00100000:1.0:1776189180.621028:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000067c9d69b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664301824:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189180.621053:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189180.621142:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57248 00000100:00100000:3.0:1776189180.621146:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009257ba59 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664301632:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189180.621155:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189180.625132:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57249 00000100:00100000:2.0:1776189180.625143:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000046db4947 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664301760:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189180.625168:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189180.625451:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664301504 00000100:00100000:0.0:1776189180.625498:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1375 00000100:00100000:0.0:1776189180.625502:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008e1954f2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664301504:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189180.626082:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000067c9d69b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664301824:12345-0@lo:38:kworker/u8:1.0 Request processed in 5050us (5996us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189180.626092:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9082 00000100:00100000:2.0:1776189180.626117:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000046db4947 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664301760:12345-0@lo:38:kworker/u8:1.0 Request processed in 972us (6072us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189180.626126:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57249 00010000:02020000:1.0:1776189180.626141:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:1.0:1776189180.626245:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189180.626249:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189180.626253:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bb8567a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664301760:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776189180.626265:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189180.626267:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189180.626269:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a9575a34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664301824:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189180.626352:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008e1954f2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664301504:12345-0@lo:400:kworker/u8:1.0 Request processed in 847us (19152us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189180.626360:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1375 00000100:00100000:1.0:1776189180.626569:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c4b16b97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664301504:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189180.628055:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009257ba59 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664301632:12345-0@lo:38:kworker/u8:1.0 Request processed in 6901us (8052us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189180.628068:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57248 00000100:00080000:1.0:1776189180.628274:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189180.628278:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189180.628283:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005355735b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664301632:0@lo:38:kworker/u8:1.0 00010000:02020000:1.0:1776189180.876374:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189180.928061:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189181.133712:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189181.388294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189181.644289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189181.901062:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189181.950958:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189182.156287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189182.412290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189182.668375:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189182.924586:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189182.974213:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189183.180280:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189183.436364:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189183.692289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189183.948327:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189183.996394:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:1.0:1776189184.206679:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189184.380053:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189184.380102:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189184.380275:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189184.380374:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654945984 00000100:00100000:0.0:1776189184.380429:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1376 00000100:00100000:0.0:1776189184.380436:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001aaffad0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654945984:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189184.380516:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001aaffad0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654945984:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 81us (474us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189184.380525:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1376 00000100:00100000:1.0:1776189184.389888:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654946048 00000100:00100000:1.0:1776189184.389973:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654946112 00000100:00100000:1.0:1776189184.390007:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9796 00000100:00100000:1.0:1776189184.390013:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f059006a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654946048:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189184.390098:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f059006a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654946048:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 85us (9996us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189184.390106:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9796 00000100:00100000:1.0:1776189184.390126:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57250 00000100:00100000:1.0:1776189184.390130:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022f2aa27 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654946112:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776189184.390146:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189184.390190:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022f2aa27 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654946112:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 41us (9899us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189184.390196:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57250 00010000:02020000:1.0:1776189184.460296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189184.701068:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006be5f924 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664302080:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189184.701138:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189184.701476:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776189184.701488:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e7e6067e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664302144:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189184.701521:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189184.701634:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664302080 00000100:00100000:2.0:1776189184.701683:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664302144 00000100:00100000:2.0:1776189184.701708:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6636 00000100:00100000:2.0:1776189184.701714:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004ae527b6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664302080:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189184.701751:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189184.701931:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6637 00000100:00100000:0.0:1776189184.701936:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000862ba2d5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664302144:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189184.701952:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189184.701989:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000862ba2d5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664302144:12345-0@lo:13:osp-pre-1-0.0 Request processed in 53us (468us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189184.701996:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6637 00000100:00100000:0.0:1776189184.702040:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e7e6067e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664302144:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189184.702162:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004ae527b6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664302080:12345-0@lo:13:osp-pre-0-0.0 Request processed in 448us (1024us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189184.702169:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6636 00000100:00100000:3.0:1776189184.702262:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006be5f924 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664302080:0@lo:13:osp-pre-0-0.0 00010000:02020000:1.0:1776189184.716313:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:1.0:1776189184.974353:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189185.023458:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189185.228463:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189185.486147:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189185.730207:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fb6b7b14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664302208:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189185.730273:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189185.730352:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664302208 00000100:00100000:0.0:1776189185.730400:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6638 00000100:00100000:0.0:1776189185.730405:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009b864aa6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664302208:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189185.730437:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189185.730489:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009b864aa6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664302208:12345-0@lo:13:osp-pre-1-1.0 Request processed in 83us (217us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189185.730497:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6638 00000100:00100000:2.0:1776189185.730611:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fb6b7b14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664302208:0@lo:13:osp-pre-1-1.0 00000100:00080000:2.0:1776189185.730693:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189185.730697:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189185.730700:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189185.730704:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9327 00000100:00080000:2.0:1776189185.730707:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00080000:2.0:1776189185.730711:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189185.730715:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000869e3fc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664302336:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189185.730728:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b43d2cdc x1862459664302400/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189185.730746:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1776189185.730751:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189185.730754:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189185.730755:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189185.730758:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9327 00000100:00080000:2.0:1776189185.730760:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00100000:3.0:1776189185.730762:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00080000:2.0:1776189185.730763:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189185.730767:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000005562bb2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664302272:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189185.730773:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000624c000a x1862459664302528/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189185.730780:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189185.730782:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189185.730784:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189185.730786:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9327 00000100:00080000:2.0:1776189185.730788:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 6s 00000100:00100000:3.0:1776189185.730790:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1776189185.730791:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189185.730801:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001065b146 x1862459664302592/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189185.730837:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006be5f924 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664302464:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189185.730908:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008242b969 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664302720:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189185.730963:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b43d2cdc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664302400:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189185.735441:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189185.735758:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000607ae435 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664302848:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189185.735810:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189185.735839:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1776189185.735846:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002c38fc42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664302656:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189185.735869:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189185.735887:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000adb1e111 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664302912:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189185.735908:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189185.735975:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664302848 00000100:00100000:2.0:1776189185.736017:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664302656 00000100:00100000:2.0:1776189185.736036:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8438 00000100:00100000:2.0:1776189185.736042:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009ea0f4b7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664302848:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189185.736095:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009ea0f4b7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664302848:12345-0@lo:400:kworker/u8:0.0 Request processed in 53us (285us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189185.736102:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8438 00000100:00100000:2.0:1776189185.736119:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8439 00000100:00100000:2.0:1776189185.736123:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a4832ec7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664302656:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189185.736147:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a4832ec7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664302656:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (278us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189185.736152:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8439 00000100:00100000:2.0:1776189185.736221:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002c38fc42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664302656:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189185.736232:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000607ae435 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664302848:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189185.736274:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664302336 00000100:00100000:2.0:1776189185.736298:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664302464 00000100:00100000:2.0:1776189185.736312:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9797 00000100:00100000:2.0:1776189185.736316:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009f0944ec pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664302336:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189185.736346:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009f0944ec pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664302336:12345-0@lo:400:kworker/u8:0.0 Request processed in 29us (5600us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189185.736351:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9797 00000100:00100000:2.0:1776189185.736361:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9798 00000100:00100000:2.0:1776189185.736364:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001b1f5bd9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664302464:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189185.736391:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001b1f5bd9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664302464:12345-0@lo:400:kworker/u8:0.0 Request processed in 26us (970us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189185.736395:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9798 00000100:00100000:2.0:1776189185.736422:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006be5f924 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664302464:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189185.736457:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664302272 00000100:00100000:2.0:1776189185.736718:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1377 00000100:00100000:2.0:1776189185.736724:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000439c1468 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664302272:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189185.736758:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000439c1468 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664302272:12345-0@lo:400:kworker/u8:0.0 Request processed in 33us (5967us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189185.736764:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1377 00000100:00100000:2.0:1776189185.736833:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664302912 00000100:00100000:2.0:1776189185.736856:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6639 00000100:00100000:2.0:1776189185.736860:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007c850b41 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664302912:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189185.736877:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189185.736908:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007c850b41 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664302912:12345-0@lo:13:osp-pre-0-1.0 Request processed in 46us (999us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189185.736913:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6639 00000100:00100000:2.0:1776189185.736938:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000adb1e111 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664302912:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189185.737772:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189185.737821:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664302720 00000100:00100000:0.0:1776189185.737872:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8440 00000100:00100000:0.0:1776189185.737877:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000073262ad2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664302720:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189185.737918:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000073262ad2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664302720:12345-0@lo:400:kworker/u8:0.0 Request processed in 40us (149us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189185.737925:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8440 00000100:00100000:0.0:1776189185.737960:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008242b969 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664302720:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189185.737974:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776189185.737983:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d8f74173 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664302784:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189185.738009:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189185.738069:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189185.738144:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000624c000a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664302528:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189185.738169:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189185.738280:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664302784 00000100:00100000:0.0:1776189185.738306:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664302400 00000100:00100000:0.0:1776189185.738319:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664302528 00000100:00100000:0.0:1776189185.738332:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9799 00000100:00100000:0.0:1776189185.738336:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000028933edf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664302784:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189185.738371:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000028933edf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664302784:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (362us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189185.738377:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9799 00000100:00100000:0.0:1776189185.738389:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57251 00000100:00100000:0.0:1776189185.738392:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cc8ba9ef pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664302400:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189185.738405:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189185.738442:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cc8ba9ef pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664302400:12345-0@lo:38:kworker/u8:0.0 Request processed in 49us (374us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189185.738449:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57251 00000100:00100000:0.0:1776189185.738461:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57252 00000100:00100000:0.0:1776189185.738464:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ee179fa1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664302528:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189185.738471:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189185.738503:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ee179fa1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664302528:12345-0@lo:38:kworker/u8:0.0 Request processed in 37us (334us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189185.738508:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57252 00000100:00100000:0.0:1776189185.738538:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d8f74173 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664302784:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189185.738573:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001065b146 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664302592:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189185.738600:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776189185.738755:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189185.738760:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189185.738764:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b43d2cdc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664302400:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776189185.738775:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189185.738778:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189185.738780:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000624c000a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664302528:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189185.738819:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664302592 00000100:00100000:0.0:1776189185.738839:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9083 00000100:00100000:0.0:1776189185.738843:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000adb1e111 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664302592:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189185.738850:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189185.739609:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000adb1e111 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664302592:12345-0@lo:38:kworker/u8:0.0 Request processed in 763us (1007us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189185.739619:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9083 00000100:00080000:0.0:1776189185.739733:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189185.739737:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189185.739740:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001065b146 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664302592:0@lo:38:kworker/u8:0.0 00010000:02020000:0.0:1776189185.746310:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189185.746524:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000005562bb2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664302272:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189185.746546:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000869e3fc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664302336:0@lo:400:kworker/u8:0.0 00010000:02020000:0.0:1776189186.004283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189186.047718:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189186.260644:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189186.516287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189186.772303:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189187.028334:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189187.068394:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189187.284342:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189187.540349:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189187.796299:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189188.052502:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189188.092393:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189188.308306:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189188.564293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189188.820393:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189189.076273:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189189.117584:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189189.332298:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189189.499739:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189189.500098:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654946304 00000100:00100000:1.0:1776189189.500150:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57253 00000100:00100000:1.0:1776189189.500157:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fb6b7b14 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654946304:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:1.0:1776189189.500207:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189189.500289:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fb6b7b14 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654946304:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 131us (557us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189189.500296:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57253 00000100:00100000:1.0:1776189189.513667:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189189.513754:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189189.513867:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654946176 00000100:00100000:0.0:1776189189.513936:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1378 00000100:00100000:0.0:1776189189.513942:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006339443e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654946176:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1776189189.514021:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006339443e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654946176:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 77us (279us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189189.514029:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1378 00000100:00100000:2.0:1776189189.514519:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654946240 00000100:00100000:2.0:1776189189.514561:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9800 00000100:00100000:2.0:1776189189.514566:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002f5aeefd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654946240:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1776189189.514619:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002f5aeefd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654946240:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 53us (960us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189189.514626:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9800 00010000:02020000:0.0:1776189189.588437:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189189.820521:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a285084b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664302976:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189189.820597:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189189.820947:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776189189.821001:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664302976 00000100:00100000:2.0:1776189189.821051:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6640 00000100:00100000:2.0:1776189189.821057:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f326e309 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664302976:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189189.821099:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189189.823290:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007df90c3e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664303040:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189189.823372:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189189.823559:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664303040 00000100:00100000:3.0:1776189189.823613:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6641 00000100:00100000:3.0:1776189189.823619:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004447f539 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664303040:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189189.823651:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189189.823690:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004447f539 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664303040:12345-0@lo:13:osp-pre-0-0.0 Request processed in 71us (319us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189189.823697:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6641 00000100:00100000:2.0:1776189189.824097:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f326e309 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664302976:12345-0@lo:13:osp-pre-1-0.0 Request processed in 3037us (3499us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189189.824106:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6640 00000100:00100000:1.0:1776189189.825273:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a285084b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664302976:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189189.825311:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007df90c3e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664303040:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189189.844301:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189190.102588:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189190.142371:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189190.356729:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189190.613336:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189190.850024:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dd5e571c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664303296:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189190.850102:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1776189190.853694:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189190.853704:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189190.853707:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189190.853716:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9332 00000100:00080000:3.0:1776189190.853720:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00080000:3.0:1776189190.853725:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189190.853765:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000db0032a x1862459664303360/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189190.855027:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776189190.855044:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005f37d44e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664303424:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189190.855114:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189190.855152:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000061a1fd5c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664303104:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189190.855199:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664303296 00000100:00100000:2.0:1776189190.855207:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189190.855227:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000869e3fc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664303168:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189190.855252:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189190.855557:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664303424 00000100:00100000:1.0:1776189190.855584:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9801 00000100:00100000:1.0:1776189190.855589:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a285084b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664303296:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189190.855732:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a285084b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664303296:12345-0@lo:400:kworker/u8:1.0 Request processed in 142us (5630us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189190.855739:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9801 00000100:00100000:1.0:1776189190.855757:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9802 00000100:00100000:1.0:1776189190.855761:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000412c3b83 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664303424:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189190.855970:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000412c3b83 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664303424:12345-0@lo:400:kworker/u8:1.0 Request processed in 206us (856us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189190.855977:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9802 00000100:00100000:1.0:1776189190.856026:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f37d44e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664303424:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189190.856039:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dd5e571c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664303296:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189190.856084:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000db0032a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664303360:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189190.856112:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189190.856137:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664303360 00000100:00100000:1.0:1776189190.856157:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57254 00000100:00100000:1.0:1776189190.856161:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e7e6067e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664303360:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189190.856198:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189190.856496:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e7e6067e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664303360:12345-0@lo:38:kworker/u8:1.0 Request processed in 333us (383us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189190.856505:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57254 00000100:00080000:3.0:1776189190.860088:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189190.860099:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189190.860102:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189190.860110:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9332 00000100:00080000:3.0:1776189190.860114:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:3.0:1776189190.860120:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189190.860160:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d0c1478c x1862459664303488/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189190.860190:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189190.860193:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189190.860194:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189190.860198:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9332 00000100:00080000:3.0:1776189190.860200:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 11s 00000100:00080000:3.0:1776189190.860202:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189190.860216:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005f435eca x1862459664303552/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189190.860313:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dd5e571c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664303808:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189190.860383:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189190.860419:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776189190.860424:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004dd53b72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664303744:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189190.860451:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189190.860489:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664303808 00000100:00100000:3.0:1776189190.860535:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8441 00000100:00100000:3.0:1776189190.860541:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f37d44e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664303808:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189190.860584:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f37d44e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664303808:12345-0@lo:400:kworker/u8:1.0 Request processed in 42us (201us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189190.860592:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8441 00000100:00100000:3.0:1776189190.860618:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664303744 00000100:00100000:3.0:1776189190.860651:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9803 00000100:00100000:3.0:1776189190.860655:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000db5ec222 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664303744:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189190.860683:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000db5ec222 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664303744:12345-0@lo:400:kworker/u8:1.0 Request processed in 28us (232us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189190.860689:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9803 00000100:00100000:3.0:1776189190.860724:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004dd53b72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664303744:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189190.860733:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dd5e571c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664303808:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189190.860762:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664303104 00000100:00100000:1.0:1776189190.862398:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d0c1478c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664303488:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189190.862467:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189190.862589:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005f435eca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664303552:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189190.862618:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776189190.862651:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189190.862655:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189190.862659:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000db0032a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664303360:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189190.862697:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664303552 00000100:00100000:1.0:1776189190.862728:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9084 00000100:00100000:1.0:1776189190.862733:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c4b16b97 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664303552:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189190.862749:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189190.862788:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c4b16b97 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664303552:12345-0@lo:38:kworker/u8:1.0 Request processed in 54us (171us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189190.862795:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9084 00000100:00100000:1.0:1776189190.862821:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664303488 00000100:00100000:1.0:1776189190.863592:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57255 00000100:00100000:3.0:1776189190.863593:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1379 00000100:00100000:1.0:1776189190.863601:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005355735b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664303488:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189190.863604:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a3281d23 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664303104:12345-0@lo:400:kworker/u8:1.0 00010000:02000000:1.0:1776189190.863615:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189190.863652:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005355735b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664303488:12345-0@lo:38:kworker/u8:1.0 Request processed in 52us (1186us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189190.863658:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57255 00000100:00080000:1.0:1776189190.863689:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189190.863693:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189190.863696:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d0c1478c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664303488:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776189190.863707:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189190.863710:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189190.863713:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f435eca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664303552:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189190.863725:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a3281d23 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664303104:12345-0@lo:400:kworker/u8:1.0 Request processed in 122us (8518us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189190.863733:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1379 00000100:00100000:1.0:1776189190.863788:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007df90c3e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664303232:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189190.863815:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189190.863836:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cf6b5d65 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664303680:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189190.863856:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189190.863872:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776189190.863877:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008ceb4933 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664303616:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189190.863893:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189190.863920:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664303680 00000100:00100000:3.0:1776189190.863958:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664303616 00000100:00100000:3.0:1776189190.863986:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8442 00000100:00100000:3.0:1776189190.863990:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006bb8567a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664303680:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189190.864029:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006bb8567a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664303680:12345-0@lo:400:kworker/u8:1.0 Request processed in 38us (172us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189190.864035:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8442 00000100:00100000:3.0:1776189190.864049:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8443 00000100:00100000:3.0:1776189190.864052:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000058fbcadb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664303616:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189190.864080:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000058fbcadb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664303616:12345-0@lo:400:kworker/u8:1.0 Request processed in 26us (186us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189190.864087:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8443 00000100:00100000:3.0:1776189190.864117:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008ceb4933 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664303616:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189190.864128:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cf6b5d65 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664303680:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189190.864204:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664303168 00000100:00100000:1.0:1776189190.864239:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664303232 00000100:00100000:1.0:1776189190.864256:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6642 00000100:00100000:1.0:1776189190.864260:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000005562bb2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664303168:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189190.864275:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189190.864305:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000005562bb2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664303168:12345-0@lo:13:osp-pre-1-1.0 Request processed in 44us (9053us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189190.864312:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6642 00000100:00100000:1.0:1776189190.864332:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6643 00000100:00100000:1.0:1776189190.864336:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a9575a34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664303232:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189190.864345:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189190.864372:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a9575a34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664303232:12345-0@lo:13:osp-pre-0-1.0 Request processed in 36us (557us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189190.864379:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6643 00000100:00100000:3.0:1776189190.864479:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007df90c3e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664303232:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189190.865567:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000061a1fd5c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664303104:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189190.865588:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000869e3fc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664303168:0@lo:13:osp-pre-1-1.0 00010000:02020000:0.0:1776189190.868277:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189191.124281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189191.167574:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189191.380442:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189191.636646:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189191.892434:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189192.148998:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189192.188818:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189192.404305:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189192.662687:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189192.916758:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189193.172328:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189193.212569:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189193.428308:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189193.684300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189193.940381:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189194.196340:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189194.236482:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189194.454684:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189194.623418:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189194.623461:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189194.623499:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189194.623630:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654946432 00000100:00100000:2.0:1776189194.623687:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654946496 00000100:00100000:2.0:1776189194.623708:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9804 00000100:00100000:2.0:1776189194.623716:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ef812603 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654946432:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189194.623755:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654946368 00000100:00100000:2.0:1776189194.623792:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ef812603 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654946432:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 76us (330us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189194.623801:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1380 00000100:00100000:2.0:1776189194.623802:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9804 00000100:00100000:3.0:1776189194.623808:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b7303d3d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654946368:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189194.623826:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57256 00000100:00100000:2.0:1776189194.623831:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000611a3c66 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654946496:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00000100:00100000:3.0:1776189194.623839:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b7303d3d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654946368:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 31us (432us total) trans 0 rc 0/0 00010000:02000000:2.0:1776189194.623850:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776189194.623850:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1380 00000100:00100000:2.0:1776189194.623884:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000611a3c66 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654946496:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 51us (384us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189194.623891:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57256 00010000:02020000:2.0:1776189194.709124:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189194.940448:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000debe3528 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664303872:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189194.940517:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189194.940626:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664303872 00000100:00100000:3.0:1776189194.940709:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6644 00000100:00100000:3.0:1776189194.940716:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000869e3fc2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664303872:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189194.940728:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007df90c3e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664303936:0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189194.940753:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189194.940762:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189194.940814:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664303936 00000100:00100000:1.0:1776189194.940839:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6645 00000100:00100000:1.0:1776189194.940843:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000917ca813 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664303936:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189194.940860:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189194.940890:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000917ca813 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664303936:12345-0@lo:13:osp-pre-0-0.0 Request processed in 46us (129us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189194.940897:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6645 00000100:00100000:1.0:1776189194.940943:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007df90c3e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664303936:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189194.943315:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000869e3fc2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664303872:12345-0@lo:13:osp-pre-1-0.0 Request processed in 2590us (2790us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189194.943330:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6644 00000100:00100000:2.0:1776189194.955561:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000debe3528 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664303872:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189194.968553:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189195.224761:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189195.260504:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189195.476964:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189195.732819:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189195.964760:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e75ea3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664304064:0@lo:400:kworker/u8:0.0 00000100:00080000:3.0:1776189195.964770:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1776189195.964771:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001065b146 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664304000:0@lo:13:osp-pre-1-1.0 00000100:00080000:3.0:1776189195.964777:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189195.964781:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189195.964786:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9337 00000100:00080000:3.0:1776189195.964790:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1776189195.964795:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189195.964815:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c89da359 x1862459664304192/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189195.964838:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189195.964841:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189195.964842:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189195.964845:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9337 00000100:00080000:3.0:1776189195.964847:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1776189195.964850:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189195.964863:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e463b423 x1862459664304384/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189195.964871:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189195.964875:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189195.964876:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1776189195.964879:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1d7bdea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664304256:0@lo:400:kworker/u8:0.0 00000100:00080000:3.0:1776189195.964881:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9337 00000100:00080000:3.0:1776189195.964885:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1776189195.964889:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189195.964900:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000c494ab9 x1862459664304448/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189195.964949:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a0c7825a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664304128:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189195.968813:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189195.968873:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe5f85f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664304512:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189195.968918:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189195.968939:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d0c1478c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664304768:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189195.968961:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189195.968978:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:2.0:1776189195.968984:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007df90c3e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664304320:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189195.969006:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189195.969028:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005f435eca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664304704:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189195.969047:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189195.969115:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664304320 00000100:00100000:1.0:1776189195.969159:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6646 00000100:00100000:1.0:1776189195.969164:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d8f74173 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664304320:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189195.969201:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189195.969256:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d8f74173 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664304320:12345-0@lo:13:osp-pre-0-1.0 Request processed in 91us (249us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189195.969262:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6646 00000100:00100000:1.0:1776189195.969288:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c89da359 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664304192:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189195.969316:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189195.969338:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e463b423 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664304384:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189195.969360:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189195.969372:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c494ab9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664304448:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189195.969393:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189195.969433:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664304192 00000100:00100000:1.0:1776189195.969452:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664304384 00000100:00100000:1.0:1776189195.969465:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57257 00000100:00100000:1.0:1776189195.969468:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000debe3528 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664304192:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189195.969481:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189195.969514:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000debe3528 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664304192:12345-0@lo:38:kworker/u8:0.0 Request processed in 44us (197us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189195.969520:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57257 00000100:00100000:1.0:1776189195.969530:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57258 00000100:00100000:1.0:1776189195.969533:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e71929fa pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664304384:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189195.969540:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189195.969866:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664304512 00000100:00100000:2.0:1776189195.969897:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664304768 00000100:00100000:2.0:1776189195.969915:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664304704 00000100:00100000:2.0:1776189195.969934:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8444 00000100:00100000:2.0:1776189195.969938:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000624c000a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664304512:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189195.974888:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189195.974953:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664304256 00000100:00100000:0.0:1776189195.974999:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9805 00000100:00100000:0.0:1776189195.975005:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000071348c99 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664304256:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189195.975054:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000071348c99 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664304256:12345-0@lo:400:kworker/u8:0.0 Request processed in 49us (175us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189195.975062:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9805 00000100:00100000:0.0:1776189195.975109:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008ceb4933 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664304640:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189195.975135:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189195.975153:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664304640 00000100:00100000:0.0:1776189195.975187:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9806 00000100:00100000:0.0:1776189195.975191:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004eaf09e3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664304640:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189195.975244:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004eaf09e3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664304640:12345-0@lo:400:kworker/u8:0.0 Request processed in 52us (108us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189195.975262:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9806 00000100:00100000:0.0:1776189195.975299:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1d7bdea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664304256:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189195.975324:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008ceb4933 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664304640:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189195.975372:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664304448 00000100:00100000:0.0:1776189195.975395:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9085 00000100:00100000:0.0:1776189195.975399:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000b229330 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664304448:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189195.975413:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189195.975445:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000b229330 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664304448:12345-0@lo:38:kworker/u8:0.0 Request processed in 45us (6052us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189195.975452:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9085 00000100:00100000:0.0:1776189195.975471:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189195.975499:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664304000 00000100:00100000:0.0:1776189195.975523:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6647 00000100:00100000:0.0:1776189195.975527:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b1d7bdea pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3363:x1862459664304000:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189195.975541:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189195.975570:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b1d7bdea pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3363:x1862459664304000:12345-0@lo:13:osp-pre-1-1.0 Request processed in 42us (99us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189195.975576:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6647 00000100:00100000:0.0:1776189195.975603:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001065b146 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664304000:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189195.975637:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8445 00000100:00100000:0.0:1776189195.975641:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b43d2cdc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664304768:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189195.975670:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b43d2cdc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664304768:12345-0@lo:400:kworker/u8:0.0 Request processed in 28us (6709us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189195.975676:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8445 00000100:00100000:0.0:1776189195.975984:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8446 00000100:00100000:0.0:1776189195.975989:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008242b969 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664304704:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189195.976024:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008242b969 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664304704:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (6977us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189195.976030:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8446 00000100:00080000:0.0:1776189195.976067:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189195.976072:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189195.976075:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c89da359 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664304192:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776189195.976086:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189195.976090:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189195.976092:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c494ab9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664304448:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189195.976166:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007df90c3e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664304320:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189195.976228:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f435eca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664304704:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189195.976238:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d0c1478c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664304768:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189195.976497:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664304064 00000100:00100000:0.0:1776189195.976532:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1381 00000100:00100000:0.0:1776189195.976543:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000089048673 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664304064:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189195.976583:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000089048673 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664304064:12345-0@lo:400:kworker/u8:0.0 Request processed in 41us (7778us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189195.976590:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1381 00000100:00100000:0.0:1776189195.976625:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e75ea3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664304064:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189195.976863:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189195.976883:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cf6b5d65 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664304576:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189195.976908:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189195.976935:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664304576 00000100:00100000:3.0:1776189195.976959:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8447 00000100:00100000:3.0:1776189195.976963:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd5e571c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664304576:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189195.976998:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd5e571c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664304576:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (89us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189195.977004:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664304128 00000100:00100000:3.0:1776189195.977005:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8447 00000100:00100000:0.0:1776189195.977036:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9807 00000100:00100000:3.0:1776189195.977041:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cf6b5d65 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664304576:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189195.977044:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000db0032a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664304128:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189195.977644:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000db0032a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664304128:12345-0@lo:400:kworker/u8:0.0 Request processed in 599us (780us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189195.977652:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9807 00000100:00100000:3.0:1776189195.979914:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a0c7825a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664304128:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189195.983872:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000624c000a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664304512:12345-0@lo:400:kworker/u8:0.0 Request processed in 13924us (14945us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189195.983888:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8444 00000100:00100000:1.0:1776189195.985331:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e71929fa pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664304384:12345-0@lo:38:kworker/u8:0.0 Request processed in 15791us (15966us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189195.985345:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57258 00000100:00080000:2.0:1776189195.985470:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189195.985475:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189195.985480:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e463b423 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664304384:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189195.985654:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe5f85f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664304512:0@lo:400:kworker/u8:0.0 00010000:02020000:2.0:1776189195.988426:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189196.244378:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189196.289406:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189196.500381:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189196.756787:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189197.015350:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189197.269404:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189197.309602:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189197.524883:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189197.780315:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189198.036523:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189198.292388:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189198.332526:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189198.548337:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189198.807193:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189199.060279:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189199.316434:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189199.356830:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189199.572356:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189199.743326:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189199.743365:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189199.743401:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189199.744593:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654946560 00000100:00100000:3.0:1776189199.744787:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1382 00000100:00100000:3.0:1776189199.744794:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000007949e4c pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654946560:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189199.744813:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654946624 00000100:00100000:1.0:1776189199.744844:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654946688 00000100:00100000:1.0:1776189199.744860:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9808 00000100:00100000:3.0:1776189199.744862:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000007949e4c pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654946560:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 69us (1543us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189199.744865:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007fe5f85f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654946624:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189199.744869:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1382 00000100:00100000:1.0:1776189199.744893:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007fe5f85f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654946624:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 28us (1529us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189199.744899:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9808 00000100:00100000:1.0:1776189199.744913:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57259 00000100:00100000:1.0:1776189199.744917:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000e75ea3b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654946688:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776189199.744931:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189199.744957:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000e75ea3b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654946688:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 39us (1556us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189199.744963:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57259 00010000:02020000:2.0:1776189199.828416:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189200.061261:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776189200.066335:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c494ab9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664304896:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189200.067993:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189200.068028:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002fbab827 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664304832:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189200.068062:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189200.068163:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664304896 00000100:00100000:3.0:1776189200.068223:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664304832 00000100:00100000:3.0:1776189200.068244:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6648 00000100:00100000:3.0:1776189200.068250:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cf6b5d65 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664304896:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189200.068281:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189200.068284:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6649 00000100:00100000:0.0:1776189200.068292:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004dd53b72 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664304832:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189200.068311:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189200.068324:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cf6b5d65 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664304896:12345-0@lo:13:osp-pre-0-0.0 Request processed in 72us (335us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189200.068332:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6648 00000100:00100000:0.0:1776189200.068350:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004dd53b72 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664304832:12345-0@lo:13:osp-pre-1-0.0 Request processed in 56us (286us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189200.068356:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6649 00000100:00100000:3.0:1776189200.068387:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c494ab9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664304896:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189200.068403:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002fbab827 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664304832:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189200.084322:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189200.340292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189200.380434:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189200.596643:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189200.852300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:3.0:1776189201.084394:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189201.084403:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189201.084406:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189201.084416:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9342 00000100:00080000:3.0:1776189201.084420:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189201.084425:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189201.084445:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008ceb4933 x1862459664305088/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189201.084479:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189201.084482:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189201.084483:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189201.084487:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9342 00000100:00080000:3.0:1776189201.084488:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189201.084491:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189201.084501:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006b6d51e7 x1862459664305216/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189201.084508:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189201.084510:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189201.084511:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189201.084514:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9342 00000100:00080000:3.0:1776189201.084516:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189201.084518:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189201.084526:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000039f4c6c3 x1862459664305280/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189201.084584:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bc3d254c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664305152:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189201.084659:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189201.084687:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000098305fef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664305536:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189201.084715:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189201.084741:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1776189201.084746:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cae4b12b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664305344:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189201.084767:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189201.084782:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000071a19d57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664305600:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189201.084801:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189201.084837:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664305152 00000100:00100000:3.0:1776189201.084882:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9809 00000100:00100000:3.0:1776189201.084889:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f12977ad pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664305152:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189201.084937:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f12977ad pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664305152:12345-0@lo:400:kworker/u8:1.0 Request processed in 48us (279us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189201.084945:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9809 00000100:00100000:3.0:1776189201.084973:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664305536 00000100:00100000:3.0:1776189201.085009:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664305344 00000100:00100000:3.0:1776189201.085023:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8448 00000100:00100000:3.0:1776189201.085026:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000516ddb7f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664305536:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189201.085056:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000516ddb7f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664305536:12345-0@lo:400:kworker/u8:1.0 Request processed in 28us (340us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189201.085062:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8448 00000100:00100000:3.0:1776189201.085075:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664305600 00000100:00100000:3.0:1776189201.085111:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8449 00000100:00100000:0.0:1776189201.085115:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005f435eca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664305024:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189201.085117:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dabf4d62 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664305344:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189201.085157:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dabf4d62 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664305344:12345-0@lo:400:kworker/u8:1.0 Request processed in 41us (390us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189201.085165:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189201.085167:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8449 00000100:00100000:0.0:1776189201.085208:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cb00f0cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664305472:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189201.085211:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8450 00000100:00100000:3.0:1776189201.085215:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ed846a61 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664305600:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189201.085243:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ed846a61 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664305600:12345-0@lo:400:kworker/u8:1.0 Request processed in 27us (441us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189201.085249:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8450 00000100:00100000:3.0:1776189201.085275:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664305024 00000100:00100000:3.0:1776189201.085298:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9810 00000100:00100000:3.0:1776189201.085303:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c89da359 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664305024:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189201.085330:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189201.085359:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c89da359 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664305024:12345-0@lo:400:kworker/u8:1.0 Request processed in 54us (195us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189201.085365:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9810 00000100:00100000:3.0:1776189201.085377:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664305472 00000100:00100000:3.0:1776189201.085396:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9811 00000100:00100000:3.0:1776189201.085399:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000c494ab9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664305472:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189201.085484:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f435eca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664305024:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189201.085502:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1776189201.085512:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d0c1478c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664304960:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189201.085567:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002fbab827 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664305664:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189201.086257:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000c494ab9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664305472:12345-0@lo:400:kworker/u8:1.0 Request processed in 853us (937us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189201.086265:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9811 00000100:00100000:3.0:1776189201.086304:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008ceb4933 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664305088:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189201.086341:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189201.086357:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006b6d51e7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664305216:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189201.086378:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189201.086390:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039f4c6c3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664305280:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189201.086410:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189201.086441:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664305088 00000100:00100000:3.0:1776189201.086462:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664305216 00000100:00100000:3.0:1776189201.086475:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57260 00000100:00100000:3.0:1776189201.086479:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003afc110b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664305088:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189201.086493:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189201.086525:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003afc110b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664305088:12345-0@lo:38:kworker/u8:1.0 Request processed in 44us (184us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189201.086530:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57260 00000100:00100000:3.0:1776189201.086554:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57261 00000100:00100000:3.0:1776189201.086557:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000094a1a2a8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664305216:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189201.086564:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189201.086587:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000094a1a2a8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664305216:12345-0@lo:38:kworker/u8:1.0 Request processed in 29us (209us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189201.086592:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57261 00000100:00080000:3.0:1776189201.086622:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189201.086625:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189201.086628:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008ceb4933 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664305088:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776189201.086647:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189201.086650:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189201.086653:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006b6d51e7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664305216:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189201.086687:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664305280 00000100:00100000:3.0:1776189201.086706:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9086 00000100:00100000:3.0:1776189201.086710:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022e717c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664305280:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189201.086717:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189201.086747:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022e717c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664305280:12345-0@lo:38:kworker/u8:1.0 Request processed in 36us (336us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189201.086752:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9086 00000100:00080000:3.0:1776189201.086773:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189201.086775:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189201.086778:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039f4c6c3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664305280:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189201.093305:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cae4b12b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664305344:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189201.093325:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189201.093328:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000071a19d57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664305600:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189201.093342:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bc3d254c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664305152:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189201.093354:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000098305fef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664305536:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189201.093371:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776189201.093382:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f34aeb18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664305728:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189201.093418:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189201.093555:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664305664 00000100:00100000:1.0:1776189201.093604:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664305728 00000100:00100000:1.0:1776189201.093622:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6650 00000100:00100000:1.0:1776189201.093627:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006be5f924 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664305664:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189201.093653:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189201.093694:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006be5f924 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664305664:12345-0@lo:13:osp-pre-1-1.0 Request processed in 66us (372us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189201.093701:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6650 00000100:00100000:1.0:1776189201.093718:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6651 00000100:00100000:2.0:1776189201.094468:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002fbab827 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664305664:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189201.094553:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189201.094581:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c9476df4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664305408:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189201.094605:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664304960 00000100:00100000:0.0:1776189201.094620:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189201.094645:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1383 00000100:00100000:2.0:1776189201.094650:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c1af929a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664304960:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189201.094652:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cb00f0cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664305472:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189201.094685:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664305408 00000100:00100000:2.0:1776189201.094697:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c1af929a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664304960:12345-0@lo:400:kworker/u8:1.0 Request processed in 46us (144us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189201.094704:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1383 00000100:00100000:0.0:1776189201.094759:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8451 00000100:00100000:0.0:1776189201.094764:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e463b423 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664305408:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189201.094797:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e463b423 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664305408:12345-0@lo:400:kworker/u8:1.0 Request processed in 32us (178us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189201.094802:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8451 00000100:00100000:0.0:1776189201.094827:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d0c1478c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664304960:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189201.094835:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c9476df4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664305408:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189201.095065:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000607ae435 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664305728:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189201.095084:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189201.095141:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000607ae435 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664305728:12345-0@lo:13:osp-pre-0-1.0 Request processed in 1418us (1722us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189201.095148:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6651 00000100:00100000:2.0:1776189201.099448:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f34aeb18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664305728:0@lo:13:osp-pre-0-1.0 00010000:02020000:2.0:1776189201.110625:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189201.364298:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189201.404464:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189201.620291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189201.876322:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189202.133666:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189202.388292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189202.428529:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189202.644535:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189202.900271:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189203.157808:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189203.412285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189203.453620:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189203.668659:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189203.924344:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189204.181623:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189204.436751:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189204.476834:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189204.692420:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189204.858274:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189204.858360:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189204.858457:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654946752 00000100:00100000:2.0:1776189204.858528:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1384 00000100:00100000:2.0:1776189204.858534:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e2219b01 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654946752:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1776189204.858624:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e2219b01 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654946752:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 88us (263us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189204.858631:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1384 00000100:00100000:3.0:1776189204.862306:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654946816 00000100:00100000:3.0:1776189204.862375:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9812 00000100:00100000:3.0:1776189204.862382:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003189c953 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654946816:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1776189204.867641:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189204.868005:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003189c953 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654946816:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 5621us (9737us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189204.868031:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9812 00000100:00100000:3.0:1776189204.868353:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654946880 00000100:00100000:3.0:1776189204.868426:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57262 00000100:00100000:3.0:1776189204.868431:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004ac5ba05 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654946880:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:3.0:1776189204.868534:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776189204.868697:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004ac5ba05 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654946880:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 264us (1063us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189204.868705:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57262 00010000:02020000:2.0:1776189204.948467:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189205.180749:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776189205.180772:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f34aeb18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664305792:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189205.180834:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189205.181056:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a0c7825a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664305856:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189205.181085:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189205.181135:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664305792 00000100:00100000:3.0:1776189205.181151:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664305856 00000100:00100000:1.0:1776189205.181219:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6652 00000100:00100000:1.0:1776189205.181225:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002c38fc42 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664305792:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189205.181257:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189205.181412:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6653 00000100:00100000:3.0:1776189205.181417:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001065b146 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664305856:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189205.181435:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189205.182452:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002c38fc42 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664305792:12345-0@lo:13:osp-pre-1-0.0 Request processed in 1222us (1614us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189205.182462:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6652 00000100:00100000:3.0:1776189205.185395:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001065b146 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664305856:12345-0@lo:13:osp-pre-0-0.0 Request processed in 3968us (4301us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189205.185416:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6653 00000100:00100000:2.0:1776189205.204138:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a0c7825a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664305856:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189205.204163:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f34aeb18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664305792:0@lo:13:osp-pre-1-0.0 00010000:02020000:0.0:1776189205.204312:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189205.460324:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189205.500953:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189205.717285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189205.972293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189206.204440:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:2.0:1776189206.204463:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f34aeb18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664305984:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189206.204536:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189206.204570:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002fbab827 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664305920:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189206.204594:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189206.204627:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664305984 00000100:00100000:2.0:1776189206.204665:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664305920 00000100:00100000:2.0:1776189206.204683:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6654 00000100:00100000:2.0:1776189206.204687:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000061a1fd5c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664305984:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189206.204708:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189206.204746:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000061a1fd5c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664305984:12345-0@lo:13:osp-pre-0-1.0 Request processed in 58us (211us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189206.204752:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6654 00000100:00100000:2.0:1776189206.204769:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6655 00000100:00100000:2.0:1776189206.204772:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001b0d11fc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664305920:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189206.204782:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189206.204806:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001b0d11fc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664305920:12345-0@lo:13:osp-pre-1-1.0 Request processed in 32us (211us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189206.204813:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6655 00000100:00100000:2.0:1776189206.204856:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f34aeb18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664305984:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189206.204868:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002fbab827 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664305920:0@lo:13:osp-pre-1-1.0 00000100:00080000:1.0:1776189206.205627:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189206.205633:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189206.205636:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189206.205642:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9348 00000100:00080000:1.0:1776189206.205646:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189206.205650:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189206.205654:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000481714e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664306112:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189206.205671:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000080eb3f20 x1862459664306176/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189206.205695:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776189206.205697:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189206.205702:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189206.205703:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189206.205708:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9348 00000100:00080000:1.0:1776189206.205710:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189206.205714:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189206.205720:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776189206.205727:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004d465759 x1862459664306304/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189206.205729:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008655ce15 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664306240:0@lo:400:kworker/u8:0.0 00000100:00080000:1.0:1776189206.205740:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189206.205743:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189206.205745:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189206.205748:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9348 00000100:00080000:1.0:1776189206.205750:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189206.205753:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189206.205758:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189206.205765:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002bf77dc0 x1862459664306368/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189206.205785:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000080eb3f20 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664306176:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189206.205815:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189206.205835:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004d465759 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664306304:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189206.205837:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664306112 00000100:00100000:3.0:1776189206.205861:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189206.205909:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003eb43bde pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664306432:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189206.205944:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189206.205962:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1776189206.205966:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f34aeb18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664306048:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189206.206003:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189206.206023:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000174c978a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664306496:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189206.206044:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189206.206068:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664306432 00000100:00100000:0.0:1776189206.206093:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664306496 00000100:00100000:0.0:1776189206.206111:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8452 00000100:00100000:0.0:1776189206.206116:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c801bdb3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664306432:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189206.206155:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c801bdb3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664306432:12345-0@lo:400:kworker/u8:0.0 Request processed in 38us (211us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189206.206161:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8452 00000100:00100000:0.0:1776189206.206380:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8453 00000100:00100000:0.0:1776189206.206384:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0c7825a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664306496:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189206.206413:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0c7825a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664306496:12345-0@lo:400:kworker/u8:0.0 Request processed in 27us (367us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189206.206418:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8453 00000100:00100000:0.0:1776189206.206438:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664306048 00000100:00100000:0.0:1776189206.206460:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1385 00000100:00100000:0.0:1776189206.206463:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003e184549 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664306048:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189206.206489:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003e184549 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664306048:12345-0@lo:400:kworker/u8:0.0 Request processed in 25us (486us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189206.206493:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1385 00000100:00100000:0.0:1776189206.206531:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f34aeb18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664306048:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189206.206540:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000174c978a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664306496:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189206.206547:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003eb43bde pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664306432:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189206.207404:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664306240 00000100:00100000:1.0:1776189206.207442:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664306176 00000100:00100000:1.0:1776189206.207452:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664306304 00000100:00100000:1.0:1776189206.207464:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9813 00000100:00100000:1.0:1776189206.207469:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c9476df4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664306112:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189206.207517:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c9476df4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664306112:12345-0@lo:400:kworker/u8:0.0 Request processed in 48us (1824us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189206.207524:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9813 00000100:00100000:1.0:1776189206.207539:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9814 00000100:00100000:1.0:1776189206.207542:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cb00f0cd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664306240:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189206.207568:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cb00f0cd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664306240:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (1811us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189206.207574:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9814 00000100:00100000:1.0:1776189206.207585:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57263 00000100:00100000:1.0:1776189206.207588:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000098305fef pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664306176:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189206.207603:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189206.207629:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000098305fef pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664306176:12345-0@lo:38:kworker/u8:0.0 Request processed in 39us (1814us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189206.207634:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57263 00000100:00100000:1.0:1776189206.207645:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57264 00000100:00100000:1.0:1776189206.207648:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bc3d254c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664306304:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189206.207655:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189206.207675:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bc3d254c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664306304:12345-0@lo:38:kworker/u8:0.0 Request processed in 25us (1816us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189206.207680:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57264 00000100:00100000:1.0:1776189206.207723:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e718565 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664306560:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189206.207752:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189206.207779:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008655ce15 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664306240:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189206.207970:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664306560 00000100:00100000:0.0:1776189206.208009:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9815 00000100:00100000:0.0:1776189206.208013:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003eb43bde pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664306560:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189206.208045:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003eb43bde pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664306560:12345-0@lo:400:kworker/u8:0.0 Request processed in 31us (293us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189206.208050:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9815 00000100:00100000:1.0:1776189206.208169:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000481714e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664306112:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189206.208261:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e718565 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664306560:0@lo:400:kworker/u8:0.0 00000100:00080000:3.0:1776189206.208763:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189206.208768:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189206.208772:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004d465759 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664306304:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189206.208778:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002bf77dc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664306368:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189206.208807:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1776189206.208846:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189206.208849:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189206.208852:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000080eb3f20 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664306176:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189206.208925:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664306368 00000100:00100000:0.0:1776189206.210832:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9087 00000100:00100000:0.0:1776189206.210841:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000071a19d57 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664306368:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189206.210860:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189206.210941:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000071a19d57 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664306368:12345-0@lo:38:kworker/u8:0.0 Request processed in 100us (2134us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189206.210949:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9087 00000100:00080000:1.0:1776189206.211044:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189206.211048:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189206.211053:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002bf77dc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664306368:0@lo:38:kworker/u8:0.0 00010000:02020000:0.0:1776189206.228499:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189206.484385:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189206.524365:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189206.740289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189206.997260:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189207.252290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189207.510913:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189207.549197:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189207.765107:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189208.020341:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189208.276282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189208.532582:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189208.574005:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189208.789086:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189209.045584:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189209.302245:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189209.556854:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189209.597090:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189209.814579:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189209.980608:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189209.980656:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189209.980687:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654947008 00000100:00100000:2.0:1776189209.980752:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9816 00000100:00100000:2.0:1776189209.980759:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002bf77dc0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654947008:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189209.980900:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002bf77dc0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654947008:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 140us (299us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189209.980918:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9816 00000100:00100000:0.0:1776189209.981507:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654946944 00000100:00100000:0.0:1776189209.981541:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1386 00000100:00100000:0.0:1776189209.981547:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000089ddaa97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654946944:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189209.981584:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000089ddaa97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654946944:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 37us (928us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189209.981592:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1386 00000100:00100000:1.0:1776189209.984047:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189209.984141:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654947072 00000100:00100000:3.0:1776189209.984202:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57265 00000100:00100000:3.0:1776189209.984208:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006e718565 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654947072:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:3.0:1776189209.984227:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776189209.984441:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006e718565 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654947072:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 232us (400us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189209.984450:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57265 00010000:02020000:0.0:1776189210.068330:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189210.300640:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776189210.300662:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000099b0e8b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664306624:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189210.300726:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189210.300822:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664306624 00000100:00100000:0.0:1776189210.300876:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6656 00000100:00100000:0.0:1776189210.300880:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cae4b12b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664306624:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776189210.300915:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189210.300962:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cae4b12b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664306624:12345-0@lo:13:osp-pre-0-0.0 Request processed in 81us (236us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189210.300970:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6656 00000100:00100000:3.0:1776189210.301161:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000099b0e8b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664306624:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189210.303310:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776189210.304532:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008655ce15 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664306688:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189210.304606:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189210.304720:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664306688 00000100:00100000:3.0:1776189210.304769:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6657 00000100:00100000:3.0:1776189210.304774:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009b5045f9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664306688:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776189210.304801:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189210.304844:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009b5045f9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664306688:12345-0@lo:13:osp-pre-1-0.0 Request processed in 69us (238us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189210.304851:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6657 00000100:00100000:2.0:1776189210.304947:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008655ce15 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664306688:0@lo:13:osp-pre-1-0.0 00010000:02020000:0.0:1776189210.325737:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189210.580400:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189210.623778:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189210.836309:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189211.093699:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:2.0:1776189211.324653:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189211.324663:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189211.324666:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189211.324674:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9353 00000100:00080000:2.0:1776189211.324677:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776189211.324682:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189211.324702:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d781d9c8 x1862459664306880/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189211.324725:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189211.324728:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189211.324729:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189211.324732:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9353 00000100:00080000:2.0:1776189211.324733:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776189211.324736:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189211.324743:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003258ba66 x1862459664307008/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189211.324751:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189211.324753:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189211.324754:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189211.324758:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9353 00000100:00080000:2.0:1776189211.324759:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776189211.324761:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189211.324772:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c0205bdc x1862459664307072/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189211.324817:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba2818a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664307136:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189211.324882:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189211.324908:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003d6663c8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664307392:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189211.324929:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189211.325020:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664307136 00000100:00100000:0.0:1776189211.325056:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664307392 00000100:00100000:0.0:1776189211.325076:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8454 00000100:00100000:2.0:1776189211.325079:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1776189211.325081:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000040b53b34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664307136:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189211.325086:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002fbab827 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664306752:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189211.325157:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8455 00000100:00100000:3.0:1776189211.325163:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e139d197 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664307392:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189211.325264:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d781d9c8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664306880:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189211.331014:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000040b53b34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664307136:12345-0@lo:400:kworker/u8:1.0 Request processed in 5924us (6133us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189211.331030:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8454 00000100:00100000:2.0:1776189211.331091:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189211.331126:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000914d0a5f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664307200:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189211.331129:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664306752 00000100:00100000:2.0:1776189211.331160:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189211.331164:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1387 00000100:00100000:0.0:1776189211.331169:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006e7c9c41 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664306752:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189211.331200:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba2818a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664307136:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189211.331220:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006e7c9c41 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664306752:12345-0@lo:400:kworker/u8:1.0 Request processed in 50us (131us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189211.331220:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776189211.331226:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1387 00000100:00100000:2.0:1776189211.331230:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000174c978a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664307456:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189211.331251:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664307200 00000100:00100000:2.0:1776189211.331252:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189211.331270:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8456 00000100:00100000:2.0:1776189211.331270:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002fbab827 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664306752:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189211.331274:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000094ac8433 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664307200:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189211.331292:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe93ff8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664306944:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189211.331306:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000094ac8433 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664307200:12345-0@lo:400:kworker/u8:1.0 Request processed in 31us (146us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189211.331311:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8456 00000100:00100000:2.0:1776189211.331326:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189211.331331:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664307456 00000100:00100000:0.0:1776189211.331351:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6658 00000100:00100000:2.0:1776189211.331351:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001b6b6588 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664307328:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189211.331355:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba2818a5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664307456:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776189211.331367:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189211.331370:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189211.331383:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:2.0:1776189211.331389:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007f9bf6ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664306816:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189211.331391:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba2818a5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664307456:12345-0@lo:13:osp-pre-0-1.0 Request processed in 36us (139us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189211.331396:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6658 00000100:00100000:2.0:1776189211.331408:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189211.331411:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664307328 00000100:00100000:2.0:1776189211.331421:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024c95b46 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664307264:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189211.331426:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8457 00000100:00100000:0.0:1776189211.331430:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000673b41e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664307328:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189211.331441:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189211.331451:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000673b41e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664307328:12345-0@lo:400:kworker/u8:1.0 Request processed in 21us (81us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189211.331452:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d9c93ad2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664307520:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189211.331456:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8457 00000100:00100000:2.0:1776189211.331471:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189211.331472:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664306944 00000100:00100000:2.0:1776189211.331489:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001b6b6588 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664307328:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189211.331491:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664306816 00000100:00100000:0.0:1776189211.331504:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664307264 00000100:00100000:2.0:1776189211.331511:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9817 00000100:00100000:2.0:1776189211.331515:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000acf34144 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664306944:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189211.331761:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000acf34144 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664306944:12345-0@lo:400:kworker/u8:1.0 Request processed in 245us (435us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189211.331855:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e139d197 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664307392:12345-0@lo:400:kworker/u8:1.0 Request processed in 6690us (6925us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189211.331862:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8455 00000100:00100000:2.0:1776189211.331948:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9817 00000100:00100000:0.0:1776189211.331951:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9818 00000100:00100000:0.0:1776189211.331955:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000aafe5128 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664306816:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189211.331960:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9819 00000100:00100000:2.0:1776189211.331964:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009e6a2c02 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664307264:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189211.331985:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000aafe5128 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664306816:12345-0@lo:400:kworker/u8:1.0 Request processed in 29us (577us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189211.331991:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9818 00000100:00100000:2.0:1776189211.331996:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009e6a2c02 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664307264:12345-0@lo:400:kworker/u8:1.0 Request processed in 32us (556us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189211.332002:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9819 00000100:00100000:0.0:1776189211.332017:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664307520 00000100:00100000:2.0:1776189211.332022:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007f9bf6ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664306816:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189211.332031:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024c95b46 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664307264:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189211.332037:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6659 00000100:00100000:2.0:1776189211.332039:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe93ff8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664306944:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189211.332040:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000887d8a72 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664307520:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189211.332050:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189211.332075:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000174c978a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664307456:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189211.332077:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000887d8a72 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664307520:12345-0@lo:13:osp-pre-1-1.0 Request processed in 36us (606us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189211.332083:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6659 00000100:00100000:2.0:1776189211.332085:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000914d0a5f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664307200:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189211.332092:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003d6663c8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664307392:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189211.332117:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d9c93ad2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664307520:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189211.332238:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189211.332253:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003258ba66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664307008:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189211.332274:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189211.332295:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c0205bdc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664307072:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189211.332464:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189211.332503:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664307072 00000100:00100000:1.0:1776189211.332526:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9088 00000100:00100000:1.0:1776189211.332530:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008655ce15 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664307072:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189211.332543:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189211.332571:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008655ce15 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664307072:12345-0@lo:38:kworker/u8:1.0 Request processed in 40us (253us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189211.332576:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9088 00000100:00080000:1.0:1776189211.332601:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189211.332605:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189211.332608:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c0205bdc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664307072:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189211.332738:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664306880 00000100:00100000:2.0:1776189211.332765:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664307008 00000100:00100000:2.0:1776189211.332779:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57266 00000100:00100000:2.0:1776189211.332783:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000029604218 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664306880:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189211.332790:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189211.332817:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000029604218 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664306880:12345-0@lo:38:kworker/u8:1.0 Request processed in 33us (580us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189211.332823:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57266 00000100:00100000:2.0:1776189211.332836:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57267 00000100:00100000:2.0:1776189211.332838:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000174c978a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664307008:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189211.332844:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189211.332864:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000174c978a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664307008:12345-0@lo:38:kworker/u8:1.0 Request processed in 24us (589us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189211.332868:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57267 00000100:00080000:1.0:1776189211.333052:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189211.333055:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189211.333059:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d781d9c8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664306880:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776189211.333067:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189211.333069:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189211.333071:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003258ba66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664307008:0@lo:38:kworker/u8:1.0 00010000:02020000:0.0:1776189211.348278:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189211.604774:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189211.646396:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189211.860704:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189212.117936:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189212.372331:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189212.628340:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189212.670644:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189212.889961:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189213.140931:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189213.397490:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189213.653370:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189213.692590:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189213.908292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189214.165055:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189214.420301:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189214.676273:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189214.720054:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189214.933650:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189215.107536:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189215.107589:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189215.107688:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654947136 00000100:00100000:0.0:1776189215.107745:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1388 00000100:00100000:0.0:1776189215.107752:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d3d96460 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654947136:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189215.107832:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d3d96460 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654947136:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 79us (242us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189215.107840:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1388 00000100:00100000:3.0:1776189215.108080:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654947264 00000100:00100000:3.0:1776189215.108110:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57268 00000100:00100000:3.0:1776189215.108115:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000080eb3f20 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654947264:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:3.0:1776189215.108128:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776189215.108155:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000080eb3f20 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654947264:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 40us (628us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189215.108161:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57268 00000100:00100000:1.0:1776189215.110372:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189215.110455:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654947200 00000100:00100000:0.0:1776189215.110510:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9820 00000100:00100000:0.0:1776189215.110516:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004d465759 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654947200:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189215.110576:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004d465759 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654947200:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 59us (211us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189215.110583:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9820 00010000:02020000:0.0:1776189215.189281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189215.422207:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc7962a4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664307584:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189215.422286:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189215.422344:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000229fc20a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664307648:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189215.422347:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664307584 00000100:00100000:3.0:1776189215.422391:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189215.422401:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664307648 00000100:00100000:2.0:1776189215.422431:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6660 00000100:00100000:2.0:1776189215.422437:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003d6663c8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664307584:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189215.422454:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6661 00000100:00100000:3.0:1776189215.422460:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000039f4c6c3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664307648:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189215.422467:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00002000:00000024:3.0:1776189215.422480:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189215.422515:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003d6663c8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664307584:12345-0@lo:13:osp-pre-1-0.0 Request processed in 77us (231us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189215.422523:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6660 00000100:00100000:3.0:1776189215.422527:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000039f4c6c3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664307648:12345-0@lo:13:osp-pre-0-0.0 Request processed in 66us (136us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189215.422535:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6661 00000100:00100000:2.0:1776189215.422574:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc7962a4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664307584:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189215.422576:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000229fc20a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664307648:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189215.444281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189215.700429:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189215.740398:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189215.956444:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189216.212857:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:2.0:1776189216.444469:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189216.444476:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189216.444483:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189216.444491:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9358 00000100:00080000:2.0:1776189216.444495:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00080000:2.0:1776189216.444500:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189216.444519:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001b6b6588 x1862459664307840/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189216.444544:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189216.444547:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189216.444549:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189216.444552:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9358 00000100:00100000:1.0:1776189216.444553:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000914d0a5f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664307712:0@lo:400:kworker/u8:0.0 00000100:00080000:2.0:1776189216.444554:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00080000:2.0:1776189216.444557:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189216.444565:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003fb63a1c x1862459664308032/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189216.444572:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189216.444575:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189216.444576:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189216.444579:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9358 00000100:00080000:2.0:1776189216.444580:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 6s 00000100:00080000:2.0:1776189216.444582:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189216.444592:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c0205bdc x1862459664308096/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189216.444626:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189216.444634:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009be3a12f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664307904:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189216.444657:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003258ba66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664308160:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189216.444792:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000db8bee4c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664307968:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189216.445967:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001b6b6588 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664307840:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189216.446021:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189216.446099:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189216.446144:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007f9bf6ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664308352:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189216.446170:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189216.447657:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189216.447698:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe93ff8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664308288:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189216.447739:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189216.447804:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664308160 00000100:00100000:2.0:1776189216.447848:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664308352 00000100:00100000:2.0:1776189216.447869:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8458 00000100:00100000:2.0:1776189216.447875:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000034c47c21 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664308160:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189216.447923:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000034c47c21 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664308160:12345-0@lo:400:kworker/u8:0.0 Request processed in 47us (1903us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189216.447929:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8458 00000100:00100000:2.0:1776189216.447946:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8459 00000100:00100000:2.0:1776189216.447949:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd47b990 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664308352:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189216.447979:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd47b990 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664308352:12345-0@lo:400:kworker/u8:0.0 Request processed in 29us (1809us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189216.447985:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8459 00000100:00100000:2.0:1776189216.448004:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664307968 00000100:00100000:2.0:1776189216.448028:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664308288 00000100:00100000:2.0:1776189216.448044:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9821 00000100:00100000:2.0:1776189216.448047:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006b6d51e7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664307968:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189216.448083:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006b6d51e7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664307968:12345-0@lo:400:kworker/u8:0.0 Request processed in 35us (1994us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189216.448102:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9821 00000100:00100000:2.0:1776189216.448114:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9822 00000100:00100000:2.0:1776189216.448117:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001d029ec5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664308288:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189216.448146:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001d029ec5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664308288:12345-0@lo:400:kworker/u8:0.0 Request processed in 28us (406us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189216.448152:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9822 00000100:00100000:0.0:1776189216.448158:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189216.448218:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003fb63a1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664308032:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189216.448245:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189216.448260:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c0205bdc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664308096:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189216.448282:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189216.448411:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664307840 00000100:00100000:2.0:1776189216.448434:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664308032 00000100:00100000:2.0:1776189216.448444:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57269 00000100:00100000:2.0:1776189216.448447:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000229fc20a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664307840:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189216.448460:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189216.448488:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000229fc20a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664307840:12345-0@lo:38:kworker/u8:0.0 Request processed in 40us (412us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189216.448494:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57269 00000100:00100000:2.0:1776189216.448506:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57270 00000100:00100000:3.0:1776189216.448506:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024c95b46 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664307776:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189216.448512:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d9c93ad2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664308032:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189216.448518:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189216.448537:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d9c93ad2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664308032:12345-0@lo:38:kworker/u8:0.0 Request processed in 26us (294us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189216.448543:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57270 00000100:00080000:0.0:1776189216.448616:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:2.0:1776189216.448619:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000db8bee4c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664307968:0@lo:400:kworker/u8:0.0 00000100:00080000:0.0:1776189216.448621:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189216.448628:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001b6b6588 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664307840:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189216.448637:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007f9bf6ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664308352:0@lo:400:kworker/u8:0.0 00000100:00080000:0.0:1776189216.448642:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189216.448645:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189216.448649:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003fb63a1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664308032:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189216.448678:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe93ff8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664308288:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189216.448693:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664308096 00000100:00100000:0.0:1776189216.448721:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9089 00000100:00100000:0.0:1776189216.448726:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000099b0e8b4 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664308096:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189216.448735:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189216.448745:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664307904 00000100:00100000:0.0:1776189216.448782:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000099b0e8b4 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664308096:12345-0@lo:38:kworker/u8:0.0 Request processed in 55us (499us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189216.448788:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9089 00000100:00100000:3.0:1776189216.448850:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189216.448864:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003258ba66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664308160:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189216.448866:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d781d9c8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664308224:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189216.448889:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189216.448902:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e463b423 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664308480:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189216.448924:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6662 00000100:00100000:3.0:1776189216.448929:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189216.448933:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c7c1f715 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664307904:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189216.448945:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664308480 00002000:00000024:2.0:1776189216.448950:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189216.448972:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6663 00000100:00100000:0.0:1776189216.448983:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a943cb24 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664308480:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189216.448993:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c7c1f715 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664307904:12345-0@lo:13:osp-pre-0-1.0 Request processed in 60us (1339us total) trans 0 rc 0/0 00002000:00000024:0.0:1776189216.448995:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189216.449004:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002fbab827 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664308416:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189216.449004:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6662 00000100:00100000:0.0:1776189216.449043:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a943cb24 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664308480:12345-0@lo:13:osp-pre-1-1.0 Request processed in 62us (114us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189216.449045:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189216.449052:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6663 00000100:00100000:1.0:1776189216.449162:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664307712 00000100:00100000:1.0:1776189216.449201:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1389 00000100:00100000:1.0:1776189216.449205:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000027854250 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664307712:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189216.449221:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664307776 00000100:00100000:1.0:1776189216.449241:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000027854250 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664307712:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (4615us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189216.449247:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1389 00000100:00100000:3.0:1776189216.449271:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9823 00000100:00100000:1.0:1776189216.449282:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000914d0a5f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664307712:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189216.449288:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000611c527e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664307776:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189216.449302:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664308224 00000100:00100000:2.0:1776189216.449325:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664308416 00000100:00100000:2.0:1776189216.449341:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8460 00000100:00100000:2.0:1776189216.449344:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c6bc9cdf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664308224:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189216.449378:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c6bc9cdf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664308224:12345-0@lo:400:kworker/u8:0.0 Request processed in 32us (488us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189216.449384:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8460 00000100:00100000:2.0:1776189216.449399:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8461 00000100:00100000:2.0:1776189216.449403:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a7b8e04c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664308416:12345-0@lo:400:kworker/u8:0.0 00000100:00080000:1.0:1776189216.449411:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189216.449416:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189216.449419:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c0205bdc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664308096:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189216.449619:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a7b8e04c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664308416:12345-0@lo:400:kworker/u8:0.0 Request processed in 215us (577us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189216.449625:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8461 00000100:00100000:2.0:1776189216.449655:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009be3a12f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664307904:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189216.449688:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d781d9c8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664308224:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189216.449700:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e463b423 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664308480:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189216.449770:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002fbab827 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664308416:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189216.450678:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000611c527e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664307776:12345-0@lo:400:kworker/u8:0.0 Request processed in 1399us (1826us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189216.450691:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9823 00000100:00100000:2.0:1776189216.454713:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024c95b46 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664307776:0@lo:400:kworker/u8:0.0 00010000:02020000:0.0:1776189216.468891:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189216.724331:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189216.764510:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189216.980306:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189217.236373:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189217.492453:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189217.748272:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189217.789068:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189218.004276:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189218.260298:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189218.516283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189218.772992:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189218.812368:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189219.028286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189219.285346:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189219.540448:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189219.796292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189219.836761:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189220.053740:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189220.222747:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189220.222814:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189220.222893:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654947392 00000100:00100000:3.0:1776189220.222968:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9824 00000100:00100000:3.0:1776189220.222974:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000f2a9137 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654947392:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189220.223046:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000f2a9137 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654947392:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 71us (307us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189220.223053:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9824 00000100:00100000:1.0:1776189220.223494:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654947328 00000100:00100000:1.0:1776189220.223558:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1390 00000100:00100000:1.0:1776189220.223563:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000192d1549 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654947328:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189220.223604:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000192d1549 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654947328:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 40us (790us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189220.223610:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1390 00000100:00100000:2.0:1776189220.225579:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189220.225669:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654947456 00000100:00100000:0.0:1776189220.225723:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57271 00000100:00100000:0.0:1776189220.225729:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a38efcb6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654947456:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776189220.225748:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189220.225812:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a38efcb6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654947456:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 82us (239us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189220.225818:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57271 00010000:02020000:2.0:1776189220.308291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189220.540767:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000030d1626f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664308544:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189220.540844:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189220.541020:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664308544 00000100:00100000:0.0:1776189220.541074:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6664 00000100:00100000:0.0:1776189220.541080:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d781d9c8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664308544:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189220.541125:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189220.541206:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d781d9c8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664308544:12345-0@lo:13:osp-pre-1-0.0 Request processed in 125us (363us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189220.541214:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6664 00000100:00100000:2.0:1776189220.541325:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000030d1626f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664308544:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189220.541608:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776189220.541626:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e463b423 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664308608:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189220.541760:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189220.541857:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664308608 00000100:00100000:0.0:1776189220.541886:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6665 00000100:00100000:0.0:1776189220.541890:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000381790a0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664308608:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776189220.541908:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189220.541944:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000381790a0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664308608:12345-0@lo:13:osp-pre-0-0.0 Request processed in 53us (185us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189220.541950:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6665 00000100:00100000:1.0:1776189220.542097:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e463b423 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664308608:0@lo:13:osp-pre-0-0.0 00010000:02020000:2.0:1776189220.564290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189220.823063:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189220.860606:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189221.076559:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189221.332289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:2.0:1776189221.567289:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189221.567297:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189221.567300:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189221.567308:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9363 00000100:00080000:2.0:1776189221.567312:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00080000:2.0:1776189221.567317:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189221.567334:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007f9bf6ea x1862459664308800/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189221.567355:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189221.567358:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189221.567360:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189221.567363:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9363 00000100:00080000:2.0:1776189221.567364:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:2.0:1776189221.567368:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189221.567376:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c0205bdc x1862459664308928/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189221.567382:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189221.567385:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189221.567386:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189221.567390:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9363 00000100:00080000:2.0:1776189221.567391:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 11s 00000100:00080000:2.0:1776189221.567394:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189221.567403:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000914d0a5f x1862459664308992/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189221.567453:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003fb63a1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664309056:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189221.567526:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189221.567554:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000723457d6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664309312:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189221.567575:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189221.567594:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1776189221.567599:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003258ba66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664308672:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189221.567626:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189221.567637:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001b6b6588 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664309120:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189221.567667:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189221.567687:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024c95b46 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664308864:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189221.567713:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189221.567728:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000050c951fb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664309248:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189221.567746:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189221.567759:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1776189221.567763:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe93ff8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664308736:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189221.567799:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189221.567815:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000db8bee4c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664309184:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189221.568011:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189221.568049:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664309056 00000100:00100000:2.0:1776189221.568083:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664309312 00000100:00100000:2.0:1776189221.568100:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664309120 00000100:00100000:2.0:1776189221.568113:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664309248 00000100:00100000:2.0:1776189221.568128:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8462 00000100:00100000:2.0:1776189221.568133:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009cace85d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664309056:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189221.568201:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009cace85d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664309056:12345-0@lo:400:kworker/u8:0.0 Request processed in 66us (674us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189221.568208:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8462 00000100:00100000:3.0:1776189221.568350:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664308672 00000100:00100000:3.0:1776189221.568378:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1391 00000100:00100000:3.0:1776189221.568382:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b64e4d66 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664308672:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189221.568416:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b64e4d66 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664308672:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (790us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189221.568423:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1391 00000100:00100000:3.0:1776189221.568450:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8463 00000100:00100000:3.0:1776189221.568454:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a680c874 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664309312:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189221.568480:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a680c874 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664309312:12345-0@lo:400:kworker/u8:0.0 Request processed in 25us (905us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189221.568484:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8463 00000100:00100000:3.0:1776189221.568497:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8464 00000100:00100000:3.0:1776189221.568500:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000073262ad2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664309120:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189221.568521:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000073262ad2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664309120:12345-0@lo:400:kworker/u8:0.0 Request processed in 21us (855us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189221.568525:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8464 00000100:00100000:3.0:1776189221.568534:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8465 00000100:00100000:3.0:1776189221.568537:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007c231a0e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664309248:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189221.568566:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007c231a0e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664309248:12345-0@lo:400:kworker/u8:0.0 Request processed in 27us (819us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189221.568571:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8465 00000100:00100000:0.0:1776189221.568753:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e463b423 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664309440:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189221.568785:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189221.568876:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664308864 00000100:00100000:3.0:1776189221.568884:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000050c951fb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664309248:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189221.568902:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664308736 00000100:00100000:0.0:1776189221.568918:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664309184 00000100:00100000:0.0:1776189221.568936:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9825 00000100:00100000:0.0:1776189221.568940:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000eeb8de0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664308864:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189221.568974:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000eeb8de0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664308864:12345-0@lo:400:kworker/u8:0.0 Request processed in 33us (1260us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189221.568980:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9825 00000100:00100000:0.0:1776189221.568991:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9826 00000100:00100000:0.0:1776189221.568995:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006bb8567a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664308736:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189221.569019:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006bb8567a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664308736:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (1221us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189221.569024:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9826 00000100:00100000:0.0:1776189221.569039:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9827 00000100:00100000:0.0:1776189221.569042:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000058fbcadb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664309184:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189221.569065:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000058fbcadb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664309184:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (1055us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189221.569070:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9827 00000100:00100000:3.0:1776189221.569091:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664309440 00000100:00100000:0.0:1776189221.569100:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe93ff8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664308736:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189221.569109:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000db8bee4c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664309184:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189221.569116:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6666 00000100:00100000:0.0:1776189221.569118:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024c95b46 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664308864:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189221.569119:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000079c74b68 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664309440:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189221.569133:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189221.569146:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003258ba66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664308672:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189221.569156:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001b6b6588 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664309120:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189221.569166:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000079c74b68 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664309440:12345-0@lo:13:osp-pre-1-1.0 Request processed in 46us (381us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189221.569168:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003fb63a1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664309056:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189221.569195:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000723457d6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664309312:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189221.569201:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6666 00000100:00100000:0.0:1776189221.569211:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776189221.569215:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005cdf5f34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664309376:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189221.569239:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189221.569325:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e463b423 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664309440:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189221.569808:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007f9bf6ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664308800:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189221.569842:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189221.569864:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c0205bdc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664308928:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189221.569885:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189221.569899:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000914d0a5f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664308992:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189221.569922:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189221.569962:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664308800 00000100:00100000:1.0:1776189221.569983:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664308928 00000100:00100000:1.0:1776189221.569998:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57272 00000100:00100000:1.0:1776189221.570001:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002e574ef8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664308800:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189221.570014:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189221.570046:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002e574ef8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664308800:12345-0@lo:38:kworker/u8:0.0 Request processed in 44us (204us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189221.570052:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57272 00000100:00100000:1.0:1776189221.570064:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57273 00000100:00100000:1.0:1776189221.570068:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003614bde1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664308928:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189221.570074:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189221.570096:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003614bde1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664308928:12345-0@lo:38:kworker/u8:0.0 Request processed in 26us (210us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189221.570100:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57273 00000100:00100000:1.0:1776189221.570117:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664308992 00000100:00100000:1.0:1776189221.570134:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9090 00000100:00100000:1.0:1776189221.570138:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000064cbacbf pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664308992:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189221.570145:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189221.570168:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000064cbacbf pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664308992:12345-0@lo:38:kworker/u8:0.0 Request processed in 28us (246us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189221.570202:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9090 00000100:00080000:1.0:1776189221.570241:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189221.570245:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189221.570249:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007f9bf6ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664308800:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776189221.570258:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189221.570261:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189221.570263:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c0205bdc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664308928:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776189221.570271:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189221.570273:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189221.570275:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000914d0a5f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664308992:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189221.572766:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664309376 00000100:00100000:3.0:1776189221.573350:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6667 00000100:00100000:3.0:1776189221.573355:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f6664e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664309376:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776189221.573379:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189221.573445:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f6664e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664309376:12345-0@lo:13:osp-pre-0-1.0 Request processed in 89us (4206us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189221.573452:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6667 00000100:00100000:0.0:1776189221.581570:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005cdf5f34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664309376:0@lo:13:osp-pre-0-1.0 00010000:02020000:2.0:1776189221.590150:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189221.844637:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189221.884739:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189222.100567:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189222.356407:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189222.612299:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189222.872094:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189222.910007:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189223.124344:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189223.380291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189223.636315:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189223.892333:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189223.932415:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189224.148347:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189224.404285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189224.660387:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189224.917030:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189224.956452:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189225.172285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189225.339896:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189225.340000:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189225.342668:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654947584 00000100:00100000:2.0:1776189225.342668:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654947648 00000100:00100000:2.0:1776189225.342810:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9828 00000100:00100000:2.0:1776189225.342818:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c0205bdc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654947584:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1776189225.342853:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57274 00000100:00100000:1.0:1776189225.342859:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000914d0a5f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654947648:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:1.0:1776189225.342874:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189225.342903:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c0205bdc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654947584:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 83us (2902us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189225.342913:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000914d0a5f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654947648:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 52us (3024us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189225.342920:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57274 00000100:00100000:2.0:1776189225.344078:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9828 00000100:00100000:2.0:1776189225.351688:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189225.352663:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654947520 00000100:00100000:1.0:1776189225.352729:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1392 00000100:00100000:1.0:1776189225.352736:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f37c27d2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654947520:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1776189225.352875:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f37c27d2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654947520:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 137us (1330us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189225.352883:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1392 00010000:02020000:2.0:1776189225.428306:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189225.660352:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776189225.660376:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003fb63a1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664309504:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189225.660381:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776189225.660394:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000030d1626f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664309568:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189225.660537:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189225.660630:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664309568 00000100:00100000:0.0:1776189225.660692:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6668 00000100:00100000:0.0:1776189225.660698:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e463b423 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664309568:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189225.660733:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189225.660787:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e463b423 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664309568:12345-0@lo:13:osp-pre-1-0.0 Request processed in 88us (251us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189225.660794:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6668 00000100:00100000:2.0:1776189225.660852:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000030d1626f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664309568:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189225.663383:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189225.663861:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664309504 00000100:00100000:0.0:1776189225.663975:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6669 00000100:00100000:0.0:1776189225.663982:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009be3a12f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664309504:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776189225.664018:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189225.664088:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009be3a12f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664309504:12345-0@lo:13:osp-pre-0-0.0 Request processed in 106us (712us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189225.664095:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6669 00000100:00100000:1.0:1776189225.664170:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003fb63a1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664309504:0@lo:13:osp-pre-0-0.0 00010000:02020000:2.0:1776189225.684599:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189225.940883:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189225.984025:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189226.196474:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189226.452346:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:2.0:1776189226.684756:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189226.684765:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189226.684769:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1776189226.684773:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00080000:2.0:1776189226.684781:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9368 00000100:00080000:2.0:1776189226.684787:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:2.0:1776189226.684805:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189226.684809:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001b6b6588 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664309696:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189226.684832:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000eccbf45c x1862459664309824/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189226.684858:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189226.684862:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189226.684863:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189226.684867:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9368 00000100:00080000:2.0:1776189226.684869:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00080000:2.0:1776189226.684873:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189226.684883:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ddb9f539 x1862459664309952/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189226.684891:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1776189226.684892:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1776189226.684897:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189226.684898:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189226.684902:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9368 00000100:00080000:2.0:1776189226.684904:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00080000:2.0:1776189226.684906:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189226.684915:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009ea0f4b7 x1862459664310016/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189226.684923:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1b59934 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664309632:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189226.684965:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e3bf25aa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664309760:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189226.685035:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664309696 00000100:00100000:1.0:1776189226.685080:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6670 00000100:00100000:1.0:1776189226.685086:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000024c95b46 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664309696:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189226.685109:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189226.685200:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189226.685225:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad48824b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664309888:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189226.685981:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189226.686007:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008242b969 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664310208:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189226.686047:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000024c95b46 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664309696:12345-0@lo:13:osp-pre-1-1.0 Request processed in 959us (1156us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189226.686057:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6670 00000100:00100000:1.0:1776189226.686087:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b43d2cdc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664310144:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189226.686114:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189226.686136:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1776189226.686142:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a4832ec7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664310080:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189226.686163:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189226.686193:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000502b3e4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664310336:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189226.686229:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189226.686254:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eccbf45c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664309824:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189226.686276:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189226.687570:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189226.687628:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664309760 00000100:00100000:2.0:1776189226.687668:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664309824 00000100:00100000:2.0:1776189226.687718:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664310208 00000100:00100000:2.0:1776189226.687737:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9829 00000100:00100000:2.0:1776189226.687755:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000020eb6bf9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664309760:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189226.687807:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000020eb6bf9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664309760:12345-0@lo:400:kworker/u8:1.0 Request processed in 51us (1830us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189226.687814:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9829 00000100:00100000:2.0:1776189226.687829:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9830 00000100:00100000:2.0:1776189226.687833:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000516ddb7f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664310208:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189226.687855:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000516ddb7f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664310208:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (288us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189226.687860:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9830 00000100:00100000:2.0:1776189226.687869:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57275 00000100:00100000:2.0:1776189226.687872:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000540d168e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664309824:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189226.687885:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189226.687907:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000540d168e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664309824:12345-0@lo:38:kworker/u8:1.0 Request processed in 34us (1630us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189226.687912:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57275 00000100:00100000:2.0:1776189226.687932:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664310336 00000100:00100000:2.0:1776189226.687951:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6671 00000100:00100000:2.0:1776189226.687954:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002fbab827 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664310336:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189226.687967:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189226.687995:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002fbab827 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664310336:12345-0@lo:13:osp-pre-0-1.0 Request processed in 40us (1767us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189226.688001:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6671 00000100:00100000:2.0:1776189226.688040:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000502b3e4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664310336:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189226.688067:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e3bf25aa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664309760:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189226.688074:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008242b969 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664310208:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189226.688118:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664310144 00000100:00100000:2.0:1776189226.688136:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664310080 00000100:00100000:2.0:1776189226.688148:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8466 00000100:00100000:2.0:1776189226.688152:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cc7962a4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664310144:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189226.688201:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cc7962a4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664310144:12345-0@lo:400:kworker/u8:1.0 Request processed in 48us (2087us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189226.688206:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8466 00000100:00100000:2.0:1776189226.688215:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8467 00000100:00100000:2.0:1776189226.688218:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000030d1626f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664310080:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189226.688238:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000030d1626f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664310080:12345-0@lo:400:kworker/u8:1.0 Request processed in 20us (2076us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189226.688243:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8467 00000100:00100000:2.0:1776189226.688265:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a4832ec7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664310080:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189226.688274:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b43d2cdc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664310144:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189226.688339:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664309632 00000100:00100000:2.0:1776189226.688369:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1393 00000100:00100000:2.0:1776189226.688372:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002e45a4e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664309632:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189226.689587:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002e45a4e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664309632:12345-0@lo:400:kworker/u8:1.0 Request processed in 1207us (4382us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189226.689600:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1393 00000100:00100000:1.0:1776189226.691908:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddb9f539 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664309952:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189226.691986:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189226.692019:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009ea0f4b7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664310016:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189226.692047:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664309952 00000100:00100000:2.0:1776189226.692088:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57276 00000100:00100000:2.0:1776189226.692094:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a109d002 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664309952:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189226.692100:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:02000000:2.0:1776189226.692112:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189226.692139:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664310016 00000100:00100000:2.0:1776189226.692153:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a109d002 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664309952:12345-0@lo:38:kworker/u8:1.0 Request processed in 58us (166us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189226.692160:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57276 00000100:00100000:1.0:1776189226.692170:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9091 00000100:00100000:1.0:1776189226.692191:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ab2fefce pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664310016:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189226.692201:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189226.692245:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ab2fefce pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664310016:12345-0@lo:38:kworker/u8:1.0 Request processed in 54us (148us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189226.692251:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9091 00000100:00100000:0.0:1776189226.692958:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1b59934 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664309632:0@lo:400:kworker/u8:1.0 00000100:00080000:1.0:1776189226.693495:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189226.693501:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189226.693506:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eccbf45c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664309824:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776189226.693518:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:0.0:1776189226.693521:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001b6b6588 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664309696:0@lo:13:osp-pre-1-1.0 00000100:00080000:1.0:1776189226.693521:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189226.693523:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddb9f539 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664309952:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776189226.693530:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189226.693532:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189226.693534:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009ea0f4b7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664310016:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189226.694112:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189226.694151:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000624c000a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664310272:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189226.694344:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664309888 00000100:00100000:3.0:1776189226.695302:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189226.695308:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9831 00000100:00100000:2.0:1776189226.695320:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d5cfaed8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664309888:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189226.695387:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d5cfaed8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664309888:12345-0@lo:400:kworker/u8:1.0 Request processed in 68us (1277us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189226.695394:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9831 00000100:00100000:3.0:1776189226.695423:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad48824b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664309888:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189226.695971:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664310272 00000100:00100000:1.0:1776189226.696017:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8468 00000100:00100000:1.0:1776189226.696021:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005cd76944 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664310272:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189226.696074:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005cd76944 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664310272:12345-0@lo:400:kworker/u8:1.0 Request processed in 51us (786us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189226.696080:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8468 00000100:00100000:3.0:1776189226.696329:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000624c000a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664310272:0@lo:400:kworker/u8:1.0 00010000:02020000:2.0:1776189226.709588:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189226.964372:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189227.004550:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189227.220294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189227.476378:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189227.732336:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189227.988253:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189228.030279:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189228.244358:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189228.501081:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189228.756287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189229.012330:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189229.052511:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189229.268290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189229.524317:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189229.780528:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189230.036587:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189230.077704:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189230.293734:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189230.462382:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189230.462438:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189230.462559:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654947840 00000100:00100000:2.0:1776189230.462610:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654947776 00000100:00100000:2.0:1776189230.462653:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9832 00000100:00100000:2.0:1776189230.462659:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd5e571c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654947776:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189230.462735:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd5e571c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654947776:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 75us (296us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189230.462743:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9832 00000100:00100000:2.0:1776189230.462760:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57277 00000100:00100000:2.0:1776189230.462763:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f37d44e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654947840:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776189230.462777:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189230.462806:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f37d44e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654947840:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 41us (432us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189230.462811:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57277 00000100:00100000:0.0:1776189230.465062:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189230.465152:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654947712 00000100:00100000:3.0:1776189230.465248:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1394 00000100:00100000:3.0:1776189230.465255:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002ea0417b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654947712:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189230.465477:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002ea0417b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654947712:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 220us (421us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189230.465484:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1394 00010000:02020000:2.0:1776189230.548523:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189230.781600:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776189230.782338:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001b6b6588 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664310464:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189230.782405:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189230.782533:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664310464 00000100:00100000:0.0:1776189230.782582:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6672 00000100:00100000:0.0:1776189230.782588:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dabf4d62 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664310464:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776189230.782618:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189230.782655:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dabf4d62 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664310464:12345-0@lo:13:osp-pre-0-0.0 Request processed in 66us (251us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189230.782662:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6672 00000100:00100000:2.0:1776189230.784246:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001b6b6588 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664310464:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189230.785514:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d0c1478c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664310400:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189230.786549:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189230.787413:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664310400 00000100:00100000:2.0:1776189230.787487:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6673 00000100:00100000:2.0:1776189230.787493:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cc4b762b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664310400:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189230.787527:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189230.787762:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cc4b762b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664310400:12345-0@lo:13:osp-pre-1-0.0 Request processed in 268us (1217us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189230.787776:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6673 00000100:00100000:3.0:1776189230.790222:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d0c1478c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664310400:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189230.804290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189231.060263:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189231.101078:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189231.316288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189231.572403:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189231.804557:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000db8bee4c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664310528:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189231.804636:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776189231.805366:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189231.805372:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189231.805375:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189231.805381:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9373 00000100:00080000:1.0:1776189231.805385:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189231.805390:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189231.805412:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d0c1478c x1862459664310784/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189231.805419:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664310528 00000100:00080000:1.0:1776189231.805464:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189231.805469:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1776189231.805472:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6674 00000100:00080000:1.0:1776189231.805472:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189231.805479:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9373 00000100:00100000:0.0:1776189231.805483:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ed846a61 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664310528:12345-0@lo:13:osp-pre-1-1.0 00000100:00080000:1.0:1776189231.805483:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189231.805489:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189231.805502:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001f927d70 x1862459664310912/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189231.805511:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189231.805514:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189231.805515:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189231.805518:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9373 00000100:00080000:1.0:1776189231.805520:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189231.805523:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00002000:00000024:0.0:1776189231.805527:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189231.805535:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f9e630d1 x1862459664310976/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189231.805579:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ed846a61 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664310528:12345-0@lo:13:osp-pre-1-1.0 Request processed in 97us (942us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189231.805586:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d0c1478c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664310784:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189231.805590:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6674 00000100:00100000:1.0:1776189231.805635:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189231.805663:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001f927d70 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664310912:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189231.805685:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189231.805738:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005f435eca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664310720:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189231.805910:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a7b82f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664310592:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189231.808355:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189231.812691:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189231.812727:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004d99e80b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664311040:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189231.812766:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189231.812799:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1776189231.812805:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008ceb4933 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664310656:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189231.812831:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189231.812855:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f34aeb18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664311104:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189231.812879:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189231.813281:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664310784 00000100:00100000:0.0:1776189231.813325:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664310912 00000100:00100000:0.0:1776189231.813356:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664310720 00000100:00100000:0.0:1776189231.813384:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9833 00000100:00100000:0.0:1776189231.813389:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dfad713d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664310720:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189231.813420:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9e630d1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664310976:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189231.813464:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189231.813506:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57278 00000100:00100000:1.0:1776189231.813511:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000154c017b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664310784:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189231.813525:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189231.813560:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000154c017b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664310784:12345-0@lo:38:kworker/u8:0.0 Request processed in 48us (7925us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189231.813568:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57278 00000100:00100000:1.0:1776189231.813587:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57279 00000100:00100000:1.0:1776189231.813591:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000058c450e1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664310912:12345-0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189231.813595:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dfad713d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664310720:12345-0@lo:400:kworker/u8:0.0 Request processed in 202us (5243us total) trans 0 rc 0/0 00010000:02000000:1.0:1776189231.813602:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189231.813607:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9833 00000100:00100000:1.0:1776189231.813643:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000058c450e1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664310912:12345-0@lo:38:kworker/u8:0.0 Request processed in 51us (7958us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189231.813649:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57279 00000100:00100000:1.0:1776189231.813669:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664310976 00000100:00100000:1.0:1776189231.813688:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9092 00000100:00100000:1.0:1776189231.813692:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004490682c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664310976:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189231.813699:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189231.813724:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004490682c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664310976:12345-0@lo:38:kworker/u8:0.0 Request processed in 31us (260us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189231.813730:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9092 00000100:00080000:0.0:1776189231.813746:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189231.813752:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189231.813758:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d0c1478c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664310784:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776189231.813770:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189231.813773:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189231.813776:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001f927d70 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664310912:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776189231.813787:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189231.813789:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189231.813793:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007df90c3e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664310848:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189231.813794:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9e630d1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664310976:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189231.813834:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664310592 00000100:00100000:1.0:1776189231.813880:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6675 00000100:00100000:1.0:1776189231.813890:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000624c000a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664310592:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189231.813905:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189231.814066:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000624c000a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664310592:12345-0@lo:13:osp-pre-0-1.0 Request processed in 180us (1381us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189231.814073:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6675 00000100:00100000:1.0:1776189231.814113:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664310656 00000100:00100000:1.0:1776189231.814138:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1395 00000100:00100000:1.0:1776189231.814142:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a9501f9f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664310656:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189231.814171:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a9501f9f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664310656:12345-0@lo:400:kworker/u8:0.0 Request processed in 28us (1339us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189231.814195:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1395 00000100:00100000:2.0:1776189231.814427:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a7b82f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664310592:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189231.814462:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008ceb4933 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664310656:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189231.814482:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664311040 00000100:00100000:0.0:1776189231.814655:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664311104 00000100:00100000:3.0:1776189231.816639:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f435eca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664310720:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189231.816657:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000048c7c2e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664311168:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189231.816707:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189231.816785:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664311168 00000100:00100000:3.0:1776189231.816826:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9834 00000100:00100000:3.0:1776189231.816831:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005a7b82f1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664311168:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189231.816890:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005a7b82f1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664311168:12345-0@lo:400:kworker/u8:0.0 Request processed in 58us (184us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189231.816897:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9834 00000100:00100000:3.0:1776189231.816932:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000048c7c2e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664311168:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189231.816969:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189231.816987:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000549bc579 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664311232:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189231.817011:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189231.821181:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664311232 00000100:00100000:3.0:1776189231.821200:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000db8bee4c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664310528:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189231.821237:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8469 00000100:00100000:1.0:1776189231.821242:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ad48824b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664311040:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189231.821245:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8470 00000100:00100000:3.0:1776189231.821250:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009ea0f4b7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664311104:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189231.821320:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ad48824b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664311040:12345-0@lo:400:kworker/u8:0.0 Request processed in 77us (8554us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189231.821327:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8469 00000100:00100000:1.0:1776189231.821343:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8471 00000100:00100000:1.0:1776189231.821347:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005cdf5f34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664311232:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189231.821379:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005cdf5f34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664311232:12345-0@lo:400:kworker/u8:0.0 Request processed in 32us (4368us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189231.821385:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8471 00000100:00100000:1.0:1776189231.821458:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004d99e80b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664311040:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189231.823965:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009ea0f4b7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664311104:12345-0@lo:400:kworker/u8:0.0 Request processed in 2711us (11082us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189231.823976:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8470 00000100:00100000:3.0:1776189231.824026:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664310848 00000100:00100000:3.0:1776189231.824074:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9835 00000100:00100000:3.0:1776189231.824079:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000502b3e4a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664310848:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189231.824111:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000502b3e4a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664310848:12345-0@lo:400:kworker/u8:0.0 Request processed in 32us (7143us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189231.824117:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9835 00000100:00100000:3.0:1776189231.824159:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007df90c3e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664310848:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189231.824188:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000549bc579 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664311232:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189231.825124:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f34aeb18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664311104:0@lo:400:kworker/u8:0.0 00010000:02020000:2.0:1776189231.828279:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189232.084323:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189232.124466:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189232.340299:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189232.596294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189232.852306:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189233.108298:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189233.148351:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189233.368424:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189233.620772:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189233.876286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189234.133413:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189234.173086:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189234.388324:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189234.645193:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189234.900293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189235.156283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189235.196579:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189235.412397:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189235.578935:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189235.578976:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189235.579025:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189235.579132:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654948032 00000100:00100000:0.0:1776189235.579204:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654947968 00000100:00100000:0.0:1776189235.579236:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9836 00000100:00100000:0.0:1776189235.579252:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000eccbf45c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654947968:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189235.579324:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000eccbf45c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654947968:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 71us (348us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189235.579332:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9836 00000100:00100000:0.0:1776189235.579357:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57280 00000100:00100000:0.0:1776189235.579361:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ddb9f539 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654948032:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776189235.579376:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189235.579402:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ddb9f539 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654948032:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 40us (475us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189235.579408:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57280 00000100:00100000:1.0:1776189235.579769:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654947904 00000100:00100000:1.0:1776189235.579808:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1396 00000100:00100000:1.0:1776189235.579812:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004c8f754f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654947904:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189235.579867:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004c8f754f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654947904:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 54us (843us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189235.579874:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1396 00010000:02020000:0.0:1776189235.668327:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189235.900956:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776189235.904835:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776189235.906327:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000db8bee4c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664311296:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189235.906436:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189235.907560:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664311296 00000100:00100000:0.0:1776189235.907611:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6676 00000100:00100000:0.0:1776189235.907618:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007df90c3e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664311296:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776189235.907675:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189235.907744:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007df90c3e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664311296:12345-0@lo:13:osp-pre-0-0.0 Request processed in 126us (1308us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189235.907751:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6676 00000100:00100000:3.0:1776189235.909440:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000db8bee4c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664311296:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189235.911823:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bd5316a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664311360:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189235.911897:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189235.912000:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664311360 00000100:00100000:0.0:1776189235.912051:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6677 00000100:00100000:0.0:1776189235.912056:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b1b59934 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664311360:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189235.912084:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189235.912475:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b1b59934 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664311360:12345-0@lo:13:osp-pre-1-0.0 Request processed in 417us (577us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189235.912482:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6677 00000100:00100000:2.0:1776189235.913825:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bd5316a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664311360:0@lo:13:osp-pre-1-0.0 00010000:02020000:0.0:1776189235.925949:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189236.180297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189236.220404:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189236.436284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189236.692286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189236.924443:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b43d2cdc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664311424:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189236.924518:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1776189236.924725:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189236.924731:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189236.924733:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189236.924738:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9378 00000100:00080000:3.0:1776189236.924741:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189236.924746:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189236.924813:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000723457d6 x1862459664311616/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189236.924833:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189236.924836:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189236.924837:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189236.924840:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9378 00000100:00080000:3.0:1776189236.924842:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189236.924845:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189236.924856:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007ec15c7b x1862459664311744/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189236.924863:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189236.924866:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189236.924867:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189236.924870:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9378 00000100:00080000:3.0:1776189236.924871:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189236.924874:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189236.924883:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006a7d5f1c x1862459664311808/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189236.925121:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f34aeb18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664311488:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189236.925160:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189236.925197:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776189236.925203:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000051b95bd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664311872:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189236.925227:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189236.925258:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664311488 00000100:00100000:1.0:1776189236.925296:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1397 00000100:00100000:1.0:1776189236.925302:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a9f52f65 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664311488:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189236.925354:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a9f52f65 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664311488:12345-0@lo:400:kworker/u8:1.0 Request processed in 51us (194us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189236.925362:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1397 00000100:00100000:1.0:1776189236.925392:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664311872 00000100:00100000:1.0:1776189236.925415:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8472 00000100:00100000:1.0:1776189236.925418:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bd5316a5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664311872:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189236.925445:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bd5316a5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664311872:12345-0@lo:400:kworker/u8:1.0 Request processed in 27us (219us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189236.925450:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8472 00000100:00100000:1.0:1776189236.925486:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000051b95bd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664311872:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189236.925494:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f34aeb18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664311488:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189236.925530:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000048c7c2e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664311552:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189236.925555:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189236.925572:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776189236.925577:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000050c951fb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664311680:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189236.925598:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189236.925624:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664311552 00000100:00100000:1.0:1776189236.925648:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664311680 00000100:00100000:1.0:1776189236.925663:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9837 00000100:00100000:1.0:1776189236.925667:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000549bc579 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664311552:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189236.925699:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000549bc579 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664311552:12345-0@lo:400:kworker/u8:1.0 Request processed in 31us (144us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189236.925705:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9837 00000100:00100000:1.0:1776189236.925717:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9838 00000100:00100000:1.0:1776189236.925720:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f34aeb18 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664311680:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189236.925745:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f34aeb18 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664311680:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (147us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189236.925750:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9838 00000100:00100000:1.0:1776189236.925779:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000050c951fb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664311680:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189236.925788:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000048c7c2e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664311552:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189236.926134:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000723457d6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664311616:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189236.926163:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189236.926200:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ec15c7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664311744:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189236.926221:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189236.926233:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006a7d5f1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664311808:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189236.926255:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189236.926283:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664311808 00000100:00100000:0.0:1776189236.926305:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9093 00000100:00100000:0.0:1776189236.926309:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001f927d70 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664311808:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189236.926332:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189236.926366:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001f927d70 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664311808:12345-0@lo:38:kworker/u8:1.0 Request processed in 56us (111us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189236.926372:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9093 00000100:00100000:0.0:1776189236.926393:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664311616 00000100:00100000:0.0:1776189236.926410:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664311744 00000100:00100000:0.0:1776189236.926424:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57281 00000100:00100000:0.0:1776189236.926427:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000db8bee4c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664311616:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189236.926435:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189236.926459:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000db8bee4c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664311616:12345-0@lo:38:kworker/u8:1.0 Request processed in 31us (296us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189236.926464:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57281 00000100:00100000:0.0:1776189236.926475:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57282 00000100:00100000:0.0:1776189236.926478:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f9e630d1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664311744:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189236.926484:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189236.926504:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f9e630d1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664311744:12345-0@lo:38:kworker/u8:1.0 Request processed in 26us (283us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189236.926509:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57282 00000100:00080000:0.0:1776189236.926538:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189236.926541:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189236.926544:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000723457d6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664311616:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776189236.926551:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189236.926554:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189236.926556:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ec15c7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664311744:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776189236.926563:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189236.926565:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189236.926567:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006a7d5f1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664311808:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189236.926588:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776189236.926595:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003fb63a1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664311936:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189236.926619:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189236.926644:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664311424 00000100:00100000:0.0:1776189236.926668:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664311936 00000100:00100000:0.0:1776189236.926685:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6678 00000100:00100000:0.0:1776189236.926689:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001b6b6588 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664311424:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776189236.926704:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189236.926736:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001b6b6588 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664311424:12345-0@lo:13:osp-pre-0-1.0 Request processed in 46us (2220us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189236.926742:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6678 00000100:00100000:0.0:1776189236.926752:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6679 00000100:00100000:0.0:1776189236.926755:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f435eca pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664311936:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189236.926763:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189236.926784:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f435eca pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664311936:12345-0@lo:13:osp-pre-1-1.0 Request processed in 28us (165us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189236.926817:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6679 00000100:00100000:0.0:1776189236.926855:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003fb63a1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664311936:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189236.926869:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b43d2cdc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664311424:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189236.930262:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba280200 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664312064:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189236.930337:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189236.930383:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776189236.930390:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f3cbce47 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664312000:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189236.930418:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189236.930453:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664312000 00000100:00100000:1.0:1776189236.930490:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8473 00000100:00100000:1.0:1776189236.930495:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000723457d6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664312000:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189236.930545:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000723457d6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664312000:12345-0@lo:400:kworker/u8:1.0 Request processed in 48us (125us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189236.930552:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8473 00000100:00100000:1.0:1776189236.930593:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f3cbce47 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664312000:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189236.930778:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664312064 00000100:00100000:3.0:1776189236.930812:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9839 00000100:00100000:3.0:1776189236.930816:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000048c7c2e3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664312064:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189236.930901:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000048c7c2e3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664312064:12345-0@lo:400:kworker/u8:1.0 Request processed in 84us (565us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189236.930907:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9839 00000100:00100000:1.0:1776189236.935600:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba280200 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664312064:0@lo:400:kworker/u8:1.0 00010000:02020000:0.0:1776189236.948813:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189237.204958:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189237.245927:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189237.460291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189237.716283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189237.972288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189238.228282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189238.268494:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189238.484296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189238.740302:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189238.996306:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189239.252292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189239.293521:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189239.510384:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189239.765736:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189240.020277:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189240.276312:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189240.317234:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189240.535171:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189240.699101:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189240.699144:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189240.699265:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189240.699304:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654948160 00000100:00100000:1.0:1776189240.699305:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654948096 00000100:00100000:3.0:1776189240.699357:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654948224 00000100:00100000:1.0:1776189240.699361:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1398 00000100:00100000:1.0:1776189240.699370:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000be36d03e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654948096:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1776189240.699377:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9840 00000100:00100000:3.0:1776189240.699383:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000045c5468b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654948160:12345-192.168.203.25@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1776189240.699703:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000045c5468b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654948160:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 319us (558us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189240.699713:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9840 00000100:00100000:3.0:1776189240.699733:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57283 00000100:00100000:3.0:1776189240.699736:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000026520930 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654948224:12345-192.168.203.25@tcp:38:kworker/u8:1.0 00010000:02000000:3.0:1776189240.699750:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189240.701170:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000be36d03e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654948096:12345-192.168.203.25@tcp:400:kworker/u8:1.0 Request processed in 1795us (2071us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189240.701203:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000026520930 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654948224:12345-192.168.203.25@tcp:38:kworker/u8:1.0 Request processed in 1464us (1937us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189240.701211:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1398 00000100:00100000:3.0:1776189240.701216:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57283 00010000:02020000:0.0:1776189240.788314:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189241.020506:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000050c951fb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664312192:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189241.020580:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189241.020602:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008ceb4933 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664312128:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189241.020639:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189241.020739:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664312192 00000100:00100000:1.0:1776189241.020830:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6680 00000100:00100000:1.0:1776189241.020837:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000087888c88 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664312192:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189241.020867:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189241.020917:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000087888c88 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664312192:12345-0@lo:13:osp-pre-1-0.0 Request processed in 79us (338us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189241.020924:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6680 00000100:00100000:3.0:1776189241.021014:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000050c951fb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664312192:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189241.029297:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664312128 00000100:00100000:0.0:1776189241.029477:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6681 00000100:00100000:0.0:1776189241.029484:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a4832ec7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664312128:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776189241.029523:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189241.029683:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a4832ec7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664312128:12345-0@lo:13:osp-pre-0-0.0 Request processed in 198us (9044us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189241.029691:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6681 00000100:00100000:3.0:1776189241.030304:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008ceb4933 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664312128:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189241.044432:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189241.300749:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189241.340430:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189241.556783:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189241.812298:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:0.0:1776189242.044451:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189242.044459:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189242.044462:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189242.044470:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9383 00000100:00080000:0.0:1776189242.044473:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776189242.044479:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189242.044510:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d0c1478c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664312256:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189242.044594:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189242.044632:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008ceb4933 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664312320:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189242.044642:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776189242.044652:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000050c951fb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664312384:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189242.044674:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189242.044715:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664312256 00000100:00100000:1.0:1776189242.044755:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1399 00000100:00100000:1.0:1776189242.044760:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000816fca97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664312256:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189242.044806:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000816fca97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664312256:12345-0@lo:400:kworker/u8:1.0 Request processed in 44us (213us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189242.044813:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1399 00000100:00100000:3.0:1776189242.044844:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189242.044993:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664312320 00000100:00100000:2.0:1776189242.045022:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9841 00000100:00100000:2.0:1776189242.045025:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006a7d5f1c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664312320:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189242.045063:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006a7d5f1c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664312320:12345-0@lo:400:kworker/u8:1.0 Request processed in 37us (391us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189242.045069:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9841 00000100:00100000:2.0:1776189242.045086:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776189242.045093:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008242b969 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664312448:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189242.045117:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189242.045422:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003fb63a1c x1862459664312512/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189242.045451:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189242.045455:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189242.045456:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189242.045459:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9383 00000100:00080000:0.0:1776189242.045461:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776189242.045464:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189242.045476:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003258ba66 x1862459664312640/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189242.045484:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189242.045487:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189242.045488:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189242.045491:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9383 00000100:00100000:2.0:1776189242.045875:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007f9bf6ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664312576:0@lo:400:kworker/u8:1.0 00000100:00080000:0.0:1776189242.045901:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 21s 00000100:00100000:2.0:1776189242.045906:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776189242.045910:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189242.045936:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d0c1478c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664312256:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189242.045965:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664312576 00000100:00100000:2.0:1776189242.045990:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9842 00000100:00100000:2.0:1776189242.045994:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b43d2cdc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664312576:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189242.046019:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664312384 00000100:00100000:2.0:1776189242.046031:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b43d2cdc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664312576:12345-0@lo:400:kworker/u8:1.0 Request processed in 36us (126us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189242.046037:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9842 00000100:00100000:2.0:1776189242.046066:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007f9bf6ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664312576:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189242.046092:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664312448 00000100:00100000:2.0:1776189242.046117:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6682 00000100:00100000:2.0:1776189242.046121:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e3bf25aa pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664312448:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189242.046136:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189242.046148:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6683 00000100:00100000:1.0:1776189242.046153:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001e34c1c7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664312384:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189242.046188:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e3bf25aa pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664312448:12345-0@lo:13:osp-pre-0-1.0 Request processed in 50us (1054us total) trans 0 rc 0/0 00002000:00000024:1.0:1776189242.046194:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189242.046197:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6682 00000100:00100000:2.0:1776189242.046244:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008242b969 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664312448:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189242.046253:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001e34c1c7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664312384:12345-0@lo:13:osp-pre-1-1.0 Request processed in 98us (1565us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189242.046261:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6683 00000100:00100000:2.0:1776189242.046281:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003fb63a1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664312512:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189242.046301:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008ceb4933 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664312320:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189242.046311:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189242.046331:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003258ba66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664312640:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189242.046344:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000050c951fb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664312384:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189242.046354:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189242.046385:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664312512 00000100:00100000:2.0:1776189242.046403:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664312640 00000100:00100000:2.0:1776189242.046417:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57284 00000100:00100000:2.0:1776189242.046420:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008242b969 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664312512:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189242.046444:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189242.046473:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57285 00000100:00100000:1.0:1776189242.046479:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008ceb4933 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664312640:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189242.046482:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008242b969 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664312512:12345-0@lo:38:kworker/u8:1.0 Request processed in 57us (169us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189242.046490:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57284 00000100:00080000:2.0:1776189242.046514:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189242.046517:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189242.046520:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003fb63a1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664312512:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189242.047631:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000fe93ff8e x1862459664312704/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189242.047687:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe93ff8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664312704:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189242.047739:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189242.047747:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001a12eabb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664312896:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189242.047783:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189242.047789:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776189242.047799:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007f9bf6ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664312768:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189242.047807:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776189242.047816:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000481714e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664312832:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189242.047832:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:02000000:1.0:1776189242.048596:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189242.048653:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008ceb4933 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664312640:12345-0@lo:38:kworker/u8:1.0 Request processed in 2172us (2299us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189242.048662:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57285 00000100:00100000:1.0:1776189242.048686:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664312896 00000100:00100000:1.0:1776189242.048719:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9843 00000100:00100000:1.0:1776189242.048724:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000279e850a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664312896:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189242.049796:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664312704 00000100:00100000:0.0:1776189242.049835:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9094 00000100:00100000:0.0:1776189242.049840:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d0c1478c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664312704:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189242.049855:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189242.049903:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d0c1478c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664312704:12345-0@lo:38:kworker/u8:1.0 Request processed in 62us (2165us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189242.049911:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9094 00000100:00100000:1.0:1776189242.049935:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000279e850a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664312896:12345-0@lo:400:kworker/u8:1.0 Request processed in 1208us (2150us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189242.049943:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9843 00000100:00080000:1.0:1776189242.049996:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189242.050001:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189242.050006:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe93ff8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664312704:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776189242.050019:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189242.050021:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189242.050025:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003258ba66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664312640:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189242.050471:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664312768 00000100:00100000:3.0:1776189242.050503:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8474 00000100:00100000:3.0:1776189242.050508:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003fb63a1c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664312768:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189242.053512:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189242.054239:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664312832 00000100:00100000:3.0:1776189242.054584:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003fb63a1c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664312768:12345-0@lo:400:kworker/u8:1.0 Request processed in 4072us (6750us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189242.054598:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8474 00000100:00100000:0.0:1776189242.054856:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8475 00000100:00100000:0.0:1776189242.054862:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007ec15c7b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664312832:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189242.056692:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001a12eabb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664312896:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189242.056853:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007f9bf6ea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664312768:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189242.057267:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007ec15c7b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664312832:12345-0@lo:400:kworker/u8:1.0 Request processed in 2403us (3760us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189242.057277:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8475 00000100:00100000:1.0:1776189242.057339:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000481714e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664312832:0@lo:400:kworker/u8:1.0 00010000:02020000:0.0:1776189242.068331:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189242.324300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189242.364497:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189242.580303:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189242.836325:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189243.093753:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189243.348270:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189243.388470:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189243.604286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189243.860409:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189244.116298:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189244.372294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189244.412407:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189244.628275:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189244.884332:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189245.141403:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189245.396316:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189245.437169:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189245.652293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189245.823620:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189245.823852:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654948416 00000100:00100000:2.0:1776189245.823934:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57286 00000100:00100000:2.0:1776189245.823940:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000051b95bd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654948416:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776189245.823960:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189245.824018:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000051b95bd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654948416:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 77us (409us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189245.824025:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57286 00000100:00100000:3.0:1776189245.824843:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189245.827283:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189245.827825:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654948288 00000100:00100000:2.0:1776189245.827883:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1400 00000100:00100000:2.0:1776189245.827890:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c4f71e57 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654948288:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189245.827955:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c4f71e57 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654948288:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 64us (679us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189245.827962:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1400 00000100:00100000:1.0:1776189245.828931:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654948352 00000100:00100000:0.0:1776189245.829150:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9844 00000100:00100000:0.0:1776189245.829158:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006080a246 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654948352:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189245.829332:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006080a246 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654948352:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 175us (4492us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189245.829340:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9844 00010000:02020000:0.0:1776189245.908426:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189246.140577:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e15ac1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664312960:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189246.140655:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189246.140992:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664312960 00000100:00100000:0.0:1776189246.141045:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6684 00000100:00100000:0.0:1776189246.141051:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000266f5a0c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664312960:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776189246.141079:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189246.141133:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000266f5a0c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664312960:12345-0@lo:13:osp-pre-0-0.0 Request processed in 80us (478us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189246.141142:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6684 00000100:00100000:2.0:1776189246.141247:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e15ac1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664312960:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189246.141524:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776189246.141534:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000481714e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664313024:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189246.141562:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189246.141656:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664313024 00000100:00100000:1.0:1776189246.141687:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6685 00000100:00100000:1.0:1776189246.141692:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d9b9c94e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664313024:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189246.141966:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189246.142092:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d9b9c94e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664313024:12345-0@lo:13:osp-pre-1-0.0 Request processed in 398us (528us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189246.142099:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6685 00000100:00100000:2.0:1776189246.144067:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000481714e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664313024:0@lo:13:osp-pre-1-0.0 00010000:02020000:0.0:1776189246.164404:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189246.422847:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189246.460575:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189246.676668:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189246.932321:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189247.164460:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776189247.164483:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001a12eabb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664313088:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189247.164550:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189247.164600:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664313088 00000100:00100000:0.0:1776189247.164645:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6686 00000100:00100000:0.0:1776189247.164650:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000074a40efb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664313088:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189247.164671:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189247.164735:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000074a40efb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664313088:12345-0@lo:13:osp-pre-1-1.0 Request processed in 75us (177us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189247.164743:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6686 00000100:00100000:0.0:1776189247.164800:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001a12eabb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664313088:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189247.164820:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace99b9b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664313152:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189247.164845:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189247.164927:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664313152 00000100:00100000:1.0:1776189247.164958:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6687 00000100:00100000:1.0:1776189247.164961:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002ed2a919 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664313152:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189247.165032:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189247.165067:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002ed2a919 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664313152:12345-0@lo:13:osp-pre-0-1.0 Request processed in 104us (222us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189247.165073:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6687 00000100:00100000:0.0:1776189247.165426:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace99b9b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664313152:0@lo:13:osp-pre-0-1.0 00000100:00080000:3.0:1776189247.165703:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189247.165707:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189247.165710:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189247.165714:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9389 00000100:00080000:3.0:1776189247.165717:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00080000:3.0:1776189247.165720:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189247.165750:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008e2382dc x1862459664313344/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189247.165772:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189247.165774:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189247.165776:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189247.165779:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9389 00000100:00080000:3.0:1776189247.165781:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00080000:3.0:1776189247.165784:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189247.165792:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000081d5cbd8 x1862459664313472/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189247.165798:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189247.165800:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189247.165802:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189247.165805:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9389 00000100:00080000:3.0:1776189247.165807:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 6s 00000100:00080000:3.0:1776189247.165809:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189247.165818:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003f67f1c5 x1862459664313536/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189247.165860:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a7066b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664313280:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189247.165894:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189247.165911:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a3202503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664313728:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189247.165927:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189247.165943:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1776189247.165947:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a78be7d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664313216:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189247.165972:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189247.165987:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002ced68f2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664313664:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189247.166009:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189247.166035:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664313280 00000100:00100000:3.0:1776189247.166065:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664313728 00000100:00100000:3.0:1776189247.166081:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9845 00000100:00100000:3.0:1776189247.166087:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000558e4f37 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664313280:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189247.166133:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000558e4f37 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664313280:12345-0@lo:400:kworker/u8:1.0 Request processed in 44us (238us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189247.166139:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9845 00000100:00100000:3.0:1776189247.166152:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9846 00000100:00100000:3.0:1776189247.166155:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000694c72a3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664313728:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189247.166212:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000694c72a3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664313728:12345-0@lo:400:kworker/u8:1.0 Request processed in 55us (284us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189247.166218:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9846 00000100:00100000:3.0:1776189247.166239:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664313216 00000100:00100000:3.0:1776189247.166259:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1401 00000100:00100000:3.0:1776189247.166262:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b1a6ecdc pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664313216:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189247.166288:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b1a6ecdc pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664313216:12345-0@lo:400:kworker/u8:1.0 Request processed in 25us (315us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189247.166294:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1401 00000100:00100000:3.0:1776189247.166327:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a78be7d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664313216:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189247.166337:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a7066b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664313280:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189247.166345:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a3202503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664313728:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189247.166500:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e2382dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664313344:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189247.166531:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189247.166551:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000081d5cbd8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664313472:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189247.166570:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189247.166581:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003f67f1c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664313536:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189247.166605:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189247.166633:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664313536 00000100:00100000:1.0:1776189247.166651:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9095 00000100:00100000:1.0:1776189247.166655:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007f9bf6ea pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664313536:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189247.166668:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189247.166699:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007f9bf6ea pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664313536:12345-0@lo:38:kworker/u8:1.0 Request processed in 44us (93us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189247.166705:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9095 00000100:00080000:1.0:1776189247.166739:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189247.166742:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189247.166746:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003f67f1c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664313536:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189247.166887:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c2578d77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664313408:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189247.166915:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189247.166942:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776189247.166948:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004f903da2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664313600:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189247.166972:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189247.166996:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664313344 00000100:00100000:2.0:1776189247.167014:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664313472 00000100:00100000:2.0:1776189247.167021:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664313408 00000100:00100000:2.0:1776189247.167034:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9847 00000100:00100000:2.0:1776189247.167037:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053c82750 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664313408:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189247.167072:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053c82750 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664313408:12345-0@lo:400:kworker/u8:1.0 Request processed in 34us (156us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189247.167078:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9847 00000100:00100000:2.0:1776189247.167091:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57287 00000100:00100000:2.0:1776189247.167094:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001a12eabb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664313344:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189247.167113:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189247.167143:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001a12eabb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664313344:12345-0@lo:38:kworker/u8:1.0 Request processed in 48us (611us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189247.167149:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57287 00000100:00100000:2.0:1776189247.167159:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57288 00000100:00100000:2.0:1776189247.167163:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000481714e2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664313472:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189247.167169:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189247.167212:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000481714e2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664313472:12345-0@lo:38:kworker/u8:1.0 Request processed in 48us (641us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189247.167217:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57288 00000100:00100000:2.0:1776189247.167246:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c2578d77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664313408:0@lo:400:kworker/u8:1.0 00000100:00080000:2.0:1776189247.167266:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189247.167269:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189247.167272:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e2382dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664313344:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776189247.167282:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189247.167284:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189247.167286:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000081d5cbd8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664313472:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189247.167489:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664313664 00000100:00100000:0.0:1776189247.167526:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664313600 00000100:00100000:0.0:1776189247.167543:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8476 00000100:00100000:0.0:1776189247.167547:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000095a4d23a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664313664:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189247.167586:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000095a4d23a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664313664:12345-0@lo:400:kworker/u8:1.0 Request processed in 38us (1577us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189247.167592:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8476 00000100:00100000:0.0:1776189247.167602:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8477 00000100:00100000:0.0:1776189247.167605:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ccb5377d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664313600:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189247.167632:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ccb5377d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664313600:12345-0@lo:400:kworker/u8:1.0 Request processed in 25us (660us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189247.167637:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8477 00000100:00100000:0.0:1776189247.167665:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002ced68f2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664313664:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189247.168133:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004f903da2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664313600:0@lo:400:kworker/u8:1.0 00010000:02020000:0.0:1776189247.188283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189247.444287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189247.484974:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189247.700977:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189247.957896:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189248.212914:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189248.468692:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189248.510061:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189248.724768:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189248.980328:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189249.236291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189249.492303:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189249.538668:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189249.748591:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189250.005132:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189250.260627:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189250.517327:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189250.556685:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189250.772340:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189250.939625:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189250.939926:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189250.940568:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189250.940746:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654948480 00000100:00100000:2.0:1776189250.940811:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1402 00000100:00100000:2.0:1776189250.940817:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000090683405 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654948480:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189250.940884:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000090683405 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654948480:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 66us (1266us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189250.940892:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1402 00000100:00100000:1.0:1776189250.941512:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654948544 00000100:00100000:1.0:1776189250.941556:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654948608 00000100:00100000:1.0:1776189250.941636:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9848 00000100:00100000:1.0:1776189250.941648:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a3202503 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654948544:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189250.941695:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a3202503 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654948544:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 53us (1770us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189250.941703:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9848 00000100:00100000:1.0:1776189250.941718:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57289 00000100:00100000:1.0:1776189250.941722:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008a7066b0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654948608:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776189250.941737:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189250.941761:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008a7066b0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654948608:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 38us (1196us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189250.941767:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57289 00010000:02020000:0.0:1776189251.028304:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189251.260899:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000895d417a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664313856:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189251.260968:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189251.261024:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776189251.261030:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003258ba66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664313792:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189251.261066:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189251.261219:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664313856 00000100:00100000:1.0:1776189251.261267:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664313792 00000100:00100000:1.0:1776189251.261290:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6688 00000100:00100000:1.0:1776189251.261295:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a78be7d4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664313856:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189251.261327:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189251.261371:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a78be7d4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664313856:12345-0@lo:13:osp-pre-0-0.0 Request processed in 75us (405us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189251.261379:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6688 00000100:00100000:1.0:1776189251.261397:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6689 00000100:00100000:1.0:1776189251.261415:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000631ae276 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664313792:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189251.261440:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189251.261465:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000631ae276 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664313792:12345-0@lo:13:osp-pre-1-0.0 Request processed in 50us (399us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189251.261471:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6689 00000100:00100000:3.0:1776189251.261878:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003258ba66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664313792:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189251.261892:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000895d417a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664313856:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189251.284285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189251.540290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189251.583239:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189251.796367:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189252.052325:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189252.284437:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776189252.284460:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000af79f780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664313920:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189252.284522:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189252.284835:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe93ff8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664313984:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189252.284863:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189252.284939:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664313920 00000100:00100000:1.0:1776189252.284994:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664313984 00000100:00100000:1.0:1776189252.285011:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6690 00000100:00100000:1.0:1776189252.285017:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000733a9e44 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664313920:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189252.285037:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189252.285083:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000733a9e44 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664313920:12345-0@lo:13:osp-pre-0-1.0 Request processed in 66us (562us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189252.285090:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6690 00000100:00100000:1.0:1776189252.285106:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6691 00000100:00100000:1.0:1776189252.285109:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001706ae06 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664313984:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189252.285119:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189252.285141:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001706ae06 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664313984:12345-0@lo:13:osp-pre-1-1.0 Request processed in 32us (279us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189252.285150:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6691 00000100:00100000:1.0:1776189252.285205:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe93ff8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664313984:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189252.285219:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000af79f780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664313920:0@lo:13:osp-pre-0-1.0 00000100:00080000:1.0:1776189252.285402:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189252.285407:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189252.285410:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189252.285414:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9394 00000100:00080000:1.0:1776189252.285418:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776189252.285421:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189252.285450:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000050c951fb x1862459664314176/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189252.285477:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189252.285480:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189252.285482:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189252.285485:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9394 00000100:00080000:1.0:1776189252.285487:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776189252.285490:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189252.285502:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004afd2049 x1862459664314304/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189252.285509:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189252.285512:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189252.285513:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189252.285516:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9394 00000100:00080000:1.0:1776189252.285518:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776189252.285520:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189252.285559:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009ea5366f x1862459664314368/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189252.285598:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000030f2672a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664314432:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189252.285774:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189252.285792:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776189252.285797:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004d99e80b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664314240:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189252.285821:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189252.285924:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003258ba66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664314112:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189252.285954:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189252.285978:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f159c11f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664314560:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189252.286009:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189252.286033:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1776189252.286038:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe93ff8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664314048:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189252.286066:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189252.286079:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000029eeaafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664314496:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189252.286098:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189252.286122:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664314048 00000100:00100000:3.0:1776189252.286149:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1403 00000100:00100000:3.0:1776189252.286153:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b90a26ba pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664314048:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189252.286208:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b90a26ba pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664314048:12345-0@lo:400:kworker/u8:1.0 Request processed in 54us (141us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189252.286215:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1403 00000100:00100000:3.0:1776189252.286244:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664314432 00000100:00100000:3.0:1776189252.286266:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664314496 00000100:00100000:3.0:1776189252.286282:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8478 00000100:00100000:3.0:1776189252.286285:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007bd06d4b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664314432:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189252.286317:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007bd06d4b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664314432:12345-0@lo:400:kworker/u8:1.0 Request processed in 31us (544us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189252.286324:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8478 00000100:00100000:3.0:1776189252.286335:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8479 00000100:00100000:3.0:1776189252.286339:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000090d63ac9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664314496:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189252.286364:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000090d63ac9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664314496:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (266us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189252.286370:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8479 00000100:00100000:3.0:1776189252.286405:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe93ff8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664314048:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189252.286415:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000029eeaafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664314496:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189252.286527:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664314240 00000100:00100000:0.0:1776189252.286557:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664314112 00000100:00100000:0.0:1776189252.286573:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664314560 00000100:00100000:0.0:1776189252.286587:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9849 00000100:00100000:0.0:1776189252.286591:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fbd6246c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664314240:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189252.286635:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fbd6246c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664314240:12345-0@lo:400:kworker/u8:1.0 Request processed in 43us (814us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189252.286641:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9849 00000100:00100000:0.0:1776189252.286653:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9850 00000100:00100000:0.0:1776189252.286657:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000047dd36ff pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664314112:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189252.286681:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000047dd36ff pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664314112:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (727us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189252.286686:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9850 00000100:00100000:0.0:1776189252.286698:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9851 00000100:00100000:0.0:1776189252.286701:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c9c38316 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664314560:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189252.286723:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c9c38316 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664314560:12345-0@lo:400:kworker/u8:1.0 Request processed in 21us (715us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189252.286728:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9851 00000100:00100000:1.0:1776189252.287208:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004d99e80b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664314240:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189252.287221:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000030f2672a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664314432:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189252.287592:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003258ba66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664314112:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189252.287605:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f159c11f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664314560:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189252.287783:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000050c951fb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664314176:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189252.287831:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189252.288104:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004afd2049 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664314304:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189252.288130:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189252.288143:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009ea5366f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664314368:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189252.288166:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189252.288228:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664314176 00000100:00100000:2.0:1776189252.288254:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664314304 00000100:00100000:2.0:1776189252.288268:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57290 00000100:00100000:2.0:1776189252.288272:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004920d098 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664314176:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189252.288290:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189252.288326:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004920d098 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664314176:12345-0@lo:38:kworker/u8:1.0 Request processed in 53us (495us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189252.288333:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57290 00000100:00100000:2.0:1776189252.288348:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57291 00000100:00100000:2.0:1776189252.288351:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000025f5718c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664314304:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189252.288357:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189252.288380:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000025f5718c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664314304:12345-0@lo:38:kworker/u8:1.0 Request processed in 27us (250us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189252.288386:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57291 00000100:00100000:2.0:1776189252.288414:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664314368 00000100:00080000:3.0:1776189252.288792:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189252.288798:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189252.288801:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000050c951fb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664314176:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776189252.288809:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189252.288812:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189252.288815:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004afd2049 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664314304:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189252.289345:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9096 00000100:00100000:2.0:1776189252.289350:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fdab0c5c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664314368:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189252.289360:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189252.289580:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fdab0c5c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664314368:12345-0@lo:38:kworker/u8:1.0 Request processed in 229us (1413us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189252.289587:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9096 00000100:00080000:0.0:1776189252.289830:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189252.289835:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189252.289838:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009ea5366f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664314368:0@lo:38:kworker/u8:1.0 00010000:02020000:0.0:1776189252.308740:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189252.565067:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189252.609370:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189252.820324:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189253.076242:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189253.332340:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189253.588301:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189253.628347:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189253.844914:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189254.100281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189254.356295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189254.612980:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189254.656799:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189254.868279:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189255.124740:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189255.380322:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189255.636297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189255.676377:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189255.892308:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189256.062462:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189256.062514:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189256.062536:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654948736 00000100:00100000:2.0:1776189256.062611:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9852 00000100:00100000:2.0:1776189256.062617:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d4966c1b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654948736:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189256.062712:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d4966c1b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654948736:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 94us (257us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189256.062720:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9852 00000100:00100000:1.0:1776189256.063143:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654948672 00000100:00100000:1.0:1776189256.063211:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1404 00000100:00100000:1.0:1776189256.063216:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ef3ea18a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654948672:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189256.063265:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ef3ea18a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654948672:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 47us (751us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189256.063271:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1404 00000100:00100000:3.0:1776189256.064642:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189256.065116:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654948800 00000100:00100000:1.0:1776189256.065160:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57292 00000100:00100000:1.0:1776189256.065166:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000aac1837c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654948800:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776189256.065208:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189256.065571:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000aac1837c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654948800:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 404us (932us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189256.065578:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57292 00010000:02020000:0.0:1776189256.148291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189256.381119:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776189256.381141:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000030f2672a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664314624:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189256.381245:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189256.381400:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664314624 00000100:00100000:1.0:1776189256.381451:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6692 00000100:00100000:1.0:1776189256.381457:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000055c58d29 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664314624:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189256.381485:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189256.381531:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000055c58d29 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664314624:12345-0@lo:13:osp-pre-1-0.0 Request processed in 73us (286us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189256.381537:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6692 00000100:00100000:1.0:1776189256.381760:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000030f2672a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664314624:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189256.381986:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776189256.381997:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d9eb44b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664314688:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189256.382071:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189256.382141:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664314688 00000100:00100000:2.0:1776189256.382202:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6693 00000100:00100000:2.0:1776189256.382206:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a6e7eea6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664314688:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189256.382224:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189256.382264:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a6e7eea6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664314688:12345-0@lo:13:osp-pre-0-0.0 Request processed in 56us (193us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189256.382270:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6693 00000100:00100000:0.0:1776189256.382462:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d9eb44b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664314688:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189256.405329:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189256.661089:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189256.700504:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189256.916331:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189257.172304:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:1.0:1776189257.404784:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189257.404791:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189257.404794:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189257.404801:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9399 00000100:00080000:1.0:1776189257.404805:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:1.0:1776189257.404811:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189257.404826:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ba6916cf x1862459664314880/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189257.404855:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189257.404869:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189257.404870:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189257.404873:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9399 00000100:00080000:1.0:1776189257.404883:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00080000:1.0:1776189257.404887:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189257.404897:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000911b4242 x1862459664315008/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189257.404905:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189257.404907:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189257.404910:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189257.404912:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9399 00000100:00080000:1.0:1776189257.404914:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00080000:1.0:1776189257.404917:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189257.404925:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009b827fed x1862459664315072/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189257.404984:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086d6a2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664315136:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189257.405036:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189257.405060:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000044d1311d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664315392:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189257.405078:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189257.405111:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1776189257.405116:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ced31f93 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664314944:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189257.405139:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189257.405152:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc84a765 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664315328:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189257.405168:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189257.405216:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664315136 00000100:00100000:1.0:1776189257.405248:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664315392 00000100:00100000:1.0:1776189257.405264:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664315328 00000100:00100000:1.0:1776189257.405282:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8480 00000100:00100000:1.0:1776189257.405288:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000032656100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664315136:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189257.405332:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000032656100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664315136:12345-0@lo:400:kworker/u8:1.0 Request processed in 44us (296us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189257.405339:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8480 00000100:00100000:1.0:1776189257.405355:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8481 00000100:00100000:1.0:1776189257.405358:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ee307f96 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664315392:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189257.405379:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ee307f96 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664315392:12345-0@lo:400:kworker/u8:1.0 Request processed in 20us (301us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189257.405385:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8481 00000100:00100000:1.0:1776189257.405394:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8482 00000100:00100000:1.0:1776189257.405398:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000027d4ac99 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664315328:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189257.405417:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000027d4ac99 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664315328:12345-0@lo:400:kworker/u8:1.0 Request processed in 19us (249us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189257.405422:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8482 00000100:00100000:1.0:1776189257.405438:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664314944 00000100:00100000:1.0:1776189257.405461:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9853 00000100:00100000:1.0:1776189257.405464:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cb607afd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664314944:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189257.405491:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cb607afd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664314944:12345-0@lo:400:kworker/u8:1.0 Request processed in 26us (352us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189257.405497:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9853 00000100:00100000:1.0:1776189257.405547:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ced31f93 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664314944:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189257.405556:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc84a765 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664315328:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189257.405564:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086d6a2c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664315136:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189257.405572:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000044d1311d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664315392:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189257.405602:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba6916cf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664314880:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189257.405624:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189257.405637:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000911b4242 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664315008:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189257.405655:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189257.405664:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b827fed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664315072:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189257.405683:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189257.405776:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664314880 00000100:00100000:0.0:1776189257.405795:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664315008 00000100:00100000:0.0:1776189257.405810:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57293 00000100:00100000:0.0:1776189257.405814:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000044d1311d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664314880:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189257.405827:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189257.405857:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000044d1311d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664314880:12345-0@lo:38:kworker/u8:1.0 Request processed in 42us (233us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189257.405869:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57293 00000100:00100000:0.0:1776189257.405889:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57294 00000100:00100000:0.0:1776189257.405893:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000086d6a2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664315008:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189257.405899:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189257.405920:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000086d6a2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664315008:12345-0@lo:38:kworker/u8:1.0 Request processed in 26us (265us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189257.405925:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57294 00000100:00100000:3.0:1776189257.406100:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000030f2672a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664314752:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189257.406133:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189257.406149:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000098a9e6f8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664315200:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189257.406326:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189257.406339:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d9eb44b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664315456:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189257.406359:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189257.406377:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:3.0:1776189257.406382:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d43194c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664314816:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189257.406400:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189257.406411:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e312a63 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664315264:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189257.406430:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189257.406439:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000087888c88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664315520:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189257.406457:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189257.406487:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664315200 00000100:00100000:3.0:1776189257.406512:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8483 00000100:00100000:3.0:1776189257.406516:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001e34c1c7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664315200:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189257.406552:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001e34c1c7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664315200:12345-0@lo:400:kworker/u8:1.0 Request processed in 35us (226us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189257.406559:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8483 00000100:00100000:3.0:1776189257.406583:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664314816 00000100:00100000:3.0:1776189257.406601:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664315264 00000100:00100000:3.0:1776189257.406620:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9854 00000100:00100000:3.0:1776189257.406623:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000631ae276 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664314816:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189257.406650:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000631ae276 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664314816:12345-0@lo:400:kworker/u8:1.0 Request processed in 26us (250us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189257.406657:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9854 00000100:00100000:3.0:1776189257.406667:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9855 00000100:00100000:3.0:1776189257.406670:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004f903da2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664315264:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189257.406690:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004f903da2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664315264:12345-0@lo:400:kworker/u8:1.0 Request processed in 19us (260us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189257.406695:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9855 00000100:00100000:3.0:1776189257.406711:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664314752 00000100:00100000:3.0:1776189257.406732:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1405 00000100:00100000:3.0:1776189257.406736:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005c923f64 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664314752:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189257.406759:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005c923f64 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664314752:12345-0@lo:400:kworker/u8:1.0 Request processed in 23us (627us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189257.406764:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1405 00000100:00100000:3.0:1776189257.406789:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664315456 00000100:00100000:3.0:1776189257.406808:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664315520 00000100:00100000:3.0:1776189257.406818:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6694 00000100:00100000:3.0:1776189257.406821:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a78be7d4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664315456:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776189257.406835:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189257.406860:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a78be7d4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664315456:12345-0@lo:13:osp-pre-0-1.0 Request processed in 38us (501us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189257.406866:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6694 00000100:00100000:3.0:1776189257.406887:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6695 00000100:00100000:3.0:1776189257.406891:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002ced68f2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664315520:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189257.406901:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189257.406922:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002ced68f2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664315520:12345-0@lo:13:osp-pre-1-1.0 Request processed in 30us (465us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189257.406931:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6695 00000100:00100000:3.0:1776189257.407042:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d43194c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664314816:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189257.407052:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e312a63 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664315264:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189257.407064:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000087888c88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664315520:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189257.407072:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000030f2672a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664314752:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189257.407080:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000098a9e6f8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664315200:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189257.407090:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d9eb44b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664315456:0@lo:13:osp-pre-0-1.0 00000100:00080000:1.0:1776189257.413070:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189257.413080:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189257.413087:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba6916cf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664314880:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776189257.413102:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189257.413105:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189257.413108:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000911b4242 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664315008:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189257.413147:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664315072 00000100:00100000:3.0:1776189257.413215:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9097 00000100:00100000:3.0:1776189257.413221:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cc84a765 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664315072:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189257.413239:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00080000:0.0:1776189257.413735:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:3.0:1776189257.413737:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cc84a765 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664315072:12345-0@lo:38:kworker/u8:1.0 Request processed in 513us (8052us total) trans 0 rc -19/-19 00000100:00080000:0.0:1776189257.413741:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189257.413747:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b827fed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664315072:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189257.413751:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9097 00010000:02020000:0.0:1776189257.429075:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189257.684369:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189257.725212:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189257.940835:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189258.196352:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189258.452287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189258.708347:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189258.748560:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189258.964276:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189259.221251:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189259.476289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189259.732288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189259.772466:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189259.991032:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189260.244278:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189260.500286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189260.756305:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189260.796397:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189261.012301:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189261.179600:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189261.183840:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654948864 00000100:00100000:1.0:1776189261.183911:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1406 00000100:00100000:1.0:1776189261.183917:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cd00b680 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654948864:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189261.183997:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cd00b680 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654948864:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 77us (4403us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189261.184005:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1406 00000100:00100000:0.0:1776189261.193137:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189261.193207:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189261.193678:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654948928 00000100:00100000:1.0:1776189261.193733:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654948992 00000100:00100000:1.0:1776189261.193753:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9856 00000100:00100000:1.0:1776189261.193758:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001be0620a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654948928:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189261.193835:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001be0620a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654948928:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 76us (706us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189261.193843:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9856 00000100:00100000:1.0:1776189261.193862:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57295 00000100:00100000:1.0:1776189261.193866:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006235ac3c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654948992:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776189261.193881:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189261.193905:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006235ac3c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654948992:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 38us (698us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189261.193914:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57295 00010000:02020000:0.0:1776189261.268516:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189261.500451:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000412a95e8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664315584:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189261.500524:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189261.500724:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000031b5a624 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664315648:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189261.500760:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189261.500847:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664315584 00000100:00100000:0.0:1776189261.500865:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664315648 00000100:00100000:3.0:1776189261.500907:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6696 00000100:00100000:3.0:1776189261.500912:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000911b4242 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664315584:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189261.500942:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189261.500982:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000911b4242 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664315584:12345-0@lo:13:osp-pre-0-0.0 Request processed in 69us (458us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189261.500994:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6696 00000100:00100000:3.0:1776189261.501012:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6697 00000100:00100000:3.0:1776189261.501016:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000014a0e5e7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664315648:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776189261.501066:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189261.501098:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000014a0e5e7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664315648:12345-0@lo:13:osp-pre-1-0.0 Request processed in 82us (340us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189261.501104:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6697 00000100:00100000:1.0:1776189261.505120:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000412a95e8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664315584:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189261.505216:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000031b5a624 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664315648:0@lo:13:osp-pre-1-0.0 00010000:02020000:0.0:1776189261.524313:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189261.781054:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189261.822741:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189262.036822:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189262.292286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189262.524806:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000087888c88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664315712:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189262.524889:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189262.524948:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664315712 00000100:00100000:3.0:1776189262.524996:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6698 00000100:00100000:3.0:1776189262.525001:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000081d5cbd8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664315712:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189262.525024:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189262.525064:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000081d5cbd8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664315712:12345-0@lo:13:osp-pre-1-1.0 Request processed in 61us (176us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189262.525071:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6698 00000100:00100000:3.0:1776189262.525114:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000087888c88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664315712:0@lo:13:osp-pre-1-1.0 00000100:00080000:1.0:1776189262.525264:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189262.525269:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189262.525272:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189262.525276:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9404 00000100:00080000:1.0:1776189262.525279:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189262.525282:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189262.525299:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a0fb2966 x1862459664315904/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189262.525319:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189262.525322:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189262.525323:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189262.525326:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9404 00000100:00080000:1.0:1776189262.525337:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189262.525340:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189262.525349:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e39f5436 x1862459664316032/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189262.525360:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189262.525363:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189262.525364:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189262.525367:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9404 00000100:00080000:1.0:1776189262.525369:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189262.525372:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189262.525381:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d72c99d5 x1862459664316096/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189262.525470:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035a819c3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664315968:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189262.525500:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189262.525518:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000098a9e6f8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664316352:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189262.525537:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189262.525564:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1776189262.525569:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ced31f93 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664315840:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189262.525598:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189262.525614:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b827fed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664316288:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189262.525633:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189262.525657:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664315968 00000100:00100000:1.0:1776189262.525683:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664315840 00000100:00100000:1.0:1776189262.525696:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664316288 00000100:00100000:1.0:1776189262.525714:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9857 00000100:00100000:1.0:1776189262.525718:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000030f2672a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664315968:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189262.525750:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000030f2672a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664315968:12345-0@lo:400:kworker/u8:1.0 Request processed in 31us (251us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189262.525774:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9857 00000100:00100000:1.0:1776189262.525790:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9858 00000100:00100000:1.0:1776189262.525806:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005d43194c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664315840:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189262.526057:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9859 00000100:00100000:3.0:1776189262.526062:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009ea5366f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664316288:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189262.526093:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009ea5366f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664316288:12345-0@lo:400:kworker/u8:1.0 Request processed in 30us (459us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189262.526098:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9859 00000100:00100000:3.0:1776189262.526122:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba6916cf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664315776:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189262.526150:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189262.526203:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000033264082 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664316224:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189262.526224:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189262.526240:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776189262.526245:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000040e81793 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664316160:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189262.526264:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189262.526286:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664315776 00000100:00100000:3.0:1776189262.526311:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1407 00000100:00100000:3.0:1776189262.526314:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f6dc323d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664315776:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189262.526344:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f6dc323d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664315776:12345-0@lo:400:kworker/u8:1.0 Request processed in 29us (194us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189262.526350:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1407 00000100:00100000:3.0:1776189262.526374:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664316352 00000100:00100000:3.0:1776189262.526396:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664316224 00000100:00100000:3.0:1776189262.526409:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664316160 00000100:00100000:3.0:1776189262.526423:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8484 00000100:00100000:3.0:1776189262.526426:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008e312a63 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664316352:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189262.526453:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008e312a63 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664316352:12345-0@lo:400:kworker/u8:1.0 Request processed in 26us (915us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189262.526459:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8484 00000100:00100000:3.0:1776189262.526468:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8485 00000100:00100000:3.0:1776189262.526471:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000087888c88 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664316224:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189262.526492:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000087888c88 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664316224:12345-0@lo:400:kworker/u8:1.0 Request processed in 20us (268us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189262.526497:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8485 00000100:00100000:3.0:1776189262.526505:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8486 00000100:00100000:3.0:1776189262.526508:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008e2382dc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664316160:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189262.526528:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008e2382dc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664316160:12345-0@lo:400:kworker/u8:1.0 Request processed in 19us (264us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189262.526534:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8486 00000100:00100000:3.0:1776189262.526567:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000040e81793 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664316160:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189262.526576:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba6916cf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664315776:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189262.526583:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000033264082 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664316224:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189262.526694:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b827fed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664316288:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189262.526703:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035a819c3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664315968:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189262.526712:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000098a9e6f8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664316352:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189262.527156:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005d43194c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664315840:12345-0@lo:400:kworker/u8:1.0 Request processed in 1347us (1555us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189262.527163:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9858 00000100:00100000:3.0:1776189262.527509:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ced31f93 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664315840:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189262.527548:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a0fb2966 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664315904:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189262.527579:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189262.527589:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9c407a2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664316416:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189262.527596:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e39f5436 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664316032:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189262.527634:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189262.527635:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189262.527668:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d72c99d5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664316096:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189262.527675:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664315904 00000100:00100000:3.0:1776189262.527695:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189262.527725:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664316096 00000100:00100000:3.0:1776189262.527745:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9098 00000100:00100000:3.0:1776189262.527748:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba280200 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664316096:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189262.527782:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189262.527993:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664316032 00000100:00100000:2.0:1776189262.530539:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57296 00000100:00100000:2.0:1776189262.530550:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c2578d77 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664315904:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189262.530571:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189262.530608:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57297 00000100:00100000:0.0:1776189262.530615:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003f67f1c5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664316032:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189262.530625:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189262.530645:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c2578d77 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664315904:12345-0@lo:38:kworker/u8:1.0 Request processed in 93us (3065us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189262.530655:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57296 00000100:00100000:0.0:1776189262.530665:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003f67f1c5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664316032:12345-0@lo:38:kworker/u8:1.0 Request processed in 48us (3033us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189262.530672:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57297 00000100:00100000:0.0:1776189262.530708:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664316416 00000100:00100000:0.0:1776189262.530741:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6699 00000100:00100000:0.0:1776189262.530745:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002a43a3e2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664316416:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776189262.530774:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189262.530812:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002a43a3e2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664316416:12345-0@lo:13:osp-pre-0-1.0 Request processed in 66us (3182us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189262.530819:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6699 00000100:00100000:0.0:1776189262.531011:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9c407a2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664316416:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189262.531161:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba280200 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664316096:12345-0@lo:38:kworker/u8:1.0 Request processed in 3410us (3466us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189262.531169:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9098 00000100:00080000:2.0:1776189262.532106:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189262.532113:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189262.532119:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a0fb2966 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664315904:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776189262.532137:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189262.532140:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189262.532142:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e39f5436 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664316032:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776189262.532151:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189262.532153:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189262.532156:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d72c99d5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664316096:0@lo:38:kworker/u8:1.0 00010000:02020000:0.0:1776189262.548821:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189262.805588:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189262.845681:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189263.062899:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189263.316259:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189263.572538:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189263.828300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189263.869594:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189264.086260:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189264.341670:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189264.599838:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189264.852297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189264.892485:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189265.108305:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189265.364278:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189265.620407:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189265.876283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189265.916460:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189266.132478:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189266.298524:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189266.298584:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189266.298692:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654949120 00000100:00100000:1.0:1776189266.298740:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654949184 00000100:00100000:1.0:1776189266.298761:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9860 00000100:00100000:1.0:1776189266.298767:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f3cbce47 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654949120:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189266.298833:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f3cbce47 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654949120:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 66us (316us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189266.298840:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9860 00000100:00100000:1.0:1776189266.298858:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57298 00000100:00100000:1.0:1776189266.298861:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004afd2049 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654949184:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776189266.298876:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189266.298903:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004afd2049 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654949184:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 40us (319us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189266.298908:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57298 00000100:00100000:3.0:1776189266.304834:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189266.305223:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654949056 00000100:00100000:0.0:1776189266.305278:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1408 00000100:00100000:0.0:1776189266.305284:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000078287bc4 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654949056:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189266.305365:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000078287bc4 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654949056:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 80us (538us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189266.305372:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1408 00010000:02020000:2.0:1776189266.388288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189266.620494:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f159c11f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664316480:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189266.620572:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189266.620575:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776189266.620592:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9c407a2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664316544:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189266.620625:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189266.620632:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664316480 00000100:00100000:3.0:1776189266.620692:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6700 00000100:00100000:3.0:1776189266.620697:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000862ba2d5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664316480:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189266.620726:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189266.620768:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000862ba2d5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664316480:12345-0@lo:13:osp-pre-0-0.0 Request processed in 69us (197us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189266.620776:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6700 00000100:00100000:3.0:1776189266.620822:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f159c11f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664316480:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189266.620964:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664316544 00000100:00100000:0.0:1776189266.621044:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6701 00000100:00100000:0.0:1776189266.621049:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000748b663e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664316544:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189266.621067:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189266.621107:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000748b663e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664316544:12345-0@lo:13:osp-pre-1-0.0 Request processed in 57us (483us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189266.621114:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6701 00000100:00100000:1.0:1776189266.621196:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9c407a2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664316544:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189266.644351:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189266.900359:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189266.941000:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189267.156358:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189267.412470:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:1.0:1776189267.644366:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189267.644375:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189267.644378:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189267.644385:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9409 00000100:00080000:1.0:1776189267.644388:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189267.644392:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189267.644414:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d72c99d5 x1862459664316736/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189267.644435:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189267.644454:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189267.644455:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189267.644459:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9409 00000100:00080000:1.0:1776189267.644461:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189267.644465:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189267.644477:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a0fb2966 x1862459664316864/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189267.644485:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1776189267.644487:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000029eeaafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664316608:0@lo:400:kworker/u8:1.0 00000100:00080000:1.0:1776189267.644490:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189267.644492:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189267.644497:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9409 00000100:00080000:1.0:1776189267.644499:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189267.644503:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189267.644515:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ced31f93 x1862459664316928/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189267.644557:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189267.644590:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035a819c3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664317056:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189267.644615:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189267.644633:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000040e81793 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664317312:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189267.644651:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189267.644682:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1776189267.644687:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000098a9e6f8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664316992:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189267.644706:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189267.644717:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba6916cf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664317248:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189267.644732:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189267.644829:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664317312 00000100:00100000:1.0:1776189267.644894:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6702 00000100:00100000:1.0:1776189267.644900:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d729b5cd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664317312:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189267.644922:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189267.645262:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000040e81793 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664317312:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189267.645307:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f159c11f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664316672:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189267.645348:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189267.645377:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b827fed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664317120:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189267.645397:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189267.645416:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1776189267.645421:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e39f5436 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664316800:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189267.645496:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189267.645514:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000033264082 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664317184:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189267.645535:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189267.645643:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d72c99d5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664316736:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189267.645810:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664317056 00000100:00100000:3.0:1776189267.645842:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664316992 00000100:00100000:3.0:1776189267.645859:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664317248 00000100:00100000:3.0:1776189267.645874:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664317184 00000100:00100000:3.0:1776189267.645888:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8487 00000100:00100000:3.0:1776189267.645892:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f078b3a8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664317056:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189267.645933:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f078b3a8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664317056:12345-0@lo:400:kworker/u8:1.0 Request processed in 40us (1318us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189267.645940:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8487 00000100:00100000:3.0:1776189267.645955:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8488 00000100:00100000:3.0:1776189267.645958:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f09d90e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664316992:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189267.646069:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f09d90e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664316992:12345-0@lo:400:kworker/u8:1.0 Request processed in 109us (1362us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189267.646078:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8488 00000100:00100000:3.0:1776189267.646088:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8489 00000100:00100000:3.0:1776189267.646091:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e42142fc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664317248:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189267.646112:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e42142fc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664317248:12345-0@lo:400:kworker/u8:1.0 Request processed in 20us (1379us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189267.646116:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8489 00000100:00100000:3.0:1776189267.646124:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8490 00000100:00100000:3.0:1776189267.646126:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004d2de401 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+6:3361:x1862459664317184:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189267.646150:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004d2de401 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+6:3361:x1862459664317184:12345-0@lo:400:kworker/u8:1.0 Request processed in 23us (615us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189267.646155:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8490 00000100:00100000:1.0:1776189267.646329:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d729b5cd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664317312:12345-0@lo:13:osp-pre-0-1.0 Request processed in 63us (310us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189267.646337:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6702 00000100:00100000:1.0:1776189267.646387:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664316608 00000100:00100000:1.0:1776189267.646410:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1409 00000100:00100000:1.0:1776189267.646414:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003afac4ae pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664316608:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189267.646594:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664316672 00000100:00100000:0.0:1776189267.646621:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664317120 00000100:00100000:0.0:1776189267.646637:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664316800 00000100:00100000:0.0:1776189267.646667:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9861 00000100:00100000:0.0:1776189267.646672:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cbabaed7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664316672:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189267.646906:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9862 00000100:00100000:3.0:1776189267.646911:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001264a122 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664317120:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189267.651683:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189267.651714:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a0fb2966 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664316864:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189267.651745:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189267.651753:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ced31f93 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664316928:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189267.651773:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189267.651831:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664316736 00000100:00100000:2.0:1776189267.651868:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664316864 00000100:00100000:2.0:1776189267.651880:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9863 00000100:00100000:2.0:1776189267.651887:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0895cec pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664316800:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189267.651933:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0895cec pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664316800:12345-0@lo:400:kworker/u8:1.0 Request processed in 46us (6436us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189267.651940:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9863 00000100:00100000:2.0:1776189267.651960:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57299 00000100:00100000:2.0:1776189267.651973:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a1abf153 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664316736:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189267.651988:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189267.652020:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a1abf153 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664316736:12345-0@lo:38:kworker/u8:1.0 Request processed in 46us (343us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189267.652025:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57299 00000100:00100000:2.0:1776189267.652036:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57300 00000100:00100000:2.0:1776189267.652039:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002b22e0b2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664316864:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189267.652046:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189267.652743:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002b22e0b2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664316864:12345-0@lo:38:kworker/u8:1.0 Request processed in 25us (320us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189267.652753:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57300 00000100:00100000:2.0:1776189267.652787:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664316928 00000100:00100000:2.0:1776189267.652813:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9099 00000100:00100000:2.0:1776189267.652816:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000eaf48489 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664316928:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189267.652825:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189267.652868:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000eaf48489 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664316928:12345-0@lo:38:kworker/u8:1.0 Request processed in 51us (1095us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189267.652874:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9099 00000100:00080000:2.0:1776189267.652907:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189267.652911:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189267.652915:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d72c99d5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664316736:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776189267.652925:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189267.652927:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189267.652929:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a0fb2966 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664316864:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776189267.652937:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189267.652939:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189267.652941:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ced31f93 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664316928:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189267.653000:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000050c951fb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664317376:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189267.653030:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189267.653058:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e39f5436 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664316800:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189267.653080:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000033264082 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664317184:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189267.653107:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664317376 00000100:00100000:2.0:1776189267.653135:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6703 00000100:00100000:2.0:1776189267.653138:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000056e9ad0b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664317376:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189267.653536:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189267.653582:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000056e9ad0b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664317376:12345-0@lo:13:osp-pre-1-1.0 Request processed in 443us (552us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189267.653589:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6703 00000100:00100000:2.0:1776189267.653621:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000050c951fb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664317376:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189267.654637:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000098a9e6f8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664316992:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189267.654651:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba6916cf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664317248:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189267.654790:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035a819c3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664317056:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189267.661748:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cbabaed7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664316672:12345-0@lo:400:kworker/u8:1.0 Request processed in 15067us (16391us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189267.661763:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9861 00000100:00100000:2.0:1776189267.662113:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f159c11f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664316672:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189267.665904:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001264a122 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664317120:12345-0@lo:400:kworker/u8:1.0 Request processed in 18986us (20500us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189267.665920:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9862 00000100:00100000:2.0:1776189267.666100:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b827fed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664317120:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189267.666391:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003afac4ae pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664316608:12345-0@lo:400:kworker/u8:1.0 Request processed in 19974us (21833us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189267.666400:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1409 00000100:00100000:2.0:1776189267.666522:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000029eeaafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664316608:0@lo:400:kworker/u8:1.0 00010000:02020000:2.0:1776189267.668251:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189267.924417:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189267.965907:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189268.180279:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189268.436294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189268.692434:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189268.948348:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189268.991531:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189269.204298:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189269.460319:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189269.716751:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189269.972300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189270.012639:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189270.229204:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189270.484482:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189270.740479:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189270.996367:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189271.038546:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189271.252353:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189271.418363:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189271.418415:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189271.418540:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654949248 00000100:00100000:1.0:1776189271.418607:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1410 00000100:00100000:1.0:1776189271.418614:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000921cb3a0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654949248:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189271.418995:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654949312 00000100:00100000:3.0:1776189271.419042:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9864 00000100:00100000:3.0:1776189271.419046:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fb1c1d52 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654949312:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189271.419214:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000921cb3a0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654949248:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 598us (798us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189271.419222:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1410 00000100:00100000:3.0:1776189271.420229:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fb1c1d52 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654949312:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 1178us (1870us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189271.420242:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9864 00000100:00100000:2.0:1776189271.420450:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189271.421202:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654949376 00000100:00100000:3.0:1776189271.421243:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57301 00000100:00100000:3.0:1776189271.421249:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000403a47d8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654949376:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:3.0:1776189271.421266:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776189271.421310:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000403a47d8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654949376:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 59us (874us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189271.421317:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57301 00010000:02020000:2.0:1776189271.508284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189271.740907:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000089232900 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664317440:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189271.740988:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189271.741085:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664317440 00000100:00100000:0.0:1776189271.741132:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6704 00000100:00100000:0.0:1776189271.741138:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000023fe30ce pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664317440:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189271.741170:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189271.741244:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000023fe30ce pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664317440:12345-0@lo:13:osp-pre-1-0.0 Request processed in 106us (256us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189271.741251:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6704 00000100:00100000:3.0:1776189271.741844:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000050c951fb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664317504:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189271.742518:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000089232900 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664317440:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189271.743886:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189271.744417:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664317504 00000100:00100000:2.0:1776189271.744615:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6705 00000100:00100000:2.0:1776189271.744622:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d781d9c8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664317504:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189271.744656:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189271.744717:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d781d9c8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664317504:12345-0@lo:13:osp-pre-0-0.0 Request processed in 94us (835us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189271.744724:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6705 00000100:00100000:3.0:1776189271.744801:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000050c951fb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664317504:0@lo:13:osp-pre-0-0.0 00010000:02020000:2.0:1776189271.766680:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189272.020278:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189272.060538:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189272.276290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189272.532306:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:3.0:1776189272.764454:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189272.764462:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189272.764465:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189272.764473:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9414 00000100:00080000:3.0:1776189272.764477:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776189272.764482:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189272.764503:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000050c951fb x1862459664317696/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189272.764525:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189272.764528:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189272.764529:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189272.764533:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9414 00000100:00080000:3.0:1776189272.764534:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776189272.764537:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189272.764555:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009b864aa6 x1862459664317824/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189272.764562:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189272.764565:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189272.764567:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189272.764570:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9414 00000100:00080000:3.0:1776189272.764572:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776189272.764586:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189272.764596:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004447f539 x1862459664317888/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189272.764769:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe93ff8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664317632:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189272.764834:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189272.764868:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dabf4d62 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664318080:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189272.764887:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189272.764907:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1776189272.764912:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d4382e95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664317760:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189272.764928:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189272.764937:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed846a61 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664318144:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189272.764955:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189272.764992:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664317632 00000100:00100000:1.0:1776189272.765028:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664318080 00000100:00100000:1.0:1776189272.765039:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664317760 00000100:00100000:1.0:1776189272.765051:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9865 00000100:00100000:1.0:1776189272.765056:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000029eeaafe pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664317632:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189272.765099:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000029eeaafe pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664317632:12345-0@lo:400:kworker/u8:0.0 Request processed in 43us (266us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189272.765106:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9865 00000100:00100000:1.0:1776189272.765136:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9866 00000100:00100000:1.0:1776189272.765140:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000035a819c3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664318080:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189272.765167:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000035a819c3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664318080:12345-0@lo:400:kworker/u8:0.0 Request processed in 26us (280us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189272.765196:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9866 00000100:00100000:1.0:1776189272.765209:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9867 00000100:00100000:1.0:1776189272.765212:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba6916cf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664317760:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189272.765237:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba6916cf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664317760:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (308us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189272.765243:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9867 00000100:00100000:1.0:1776189272.765271:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664318144 00000100:00100000:1.0:1776189272.765296:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8491 00000100:00100000:1.0:1776189272.765299:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000098a9e6f8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664318144:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189272.765327:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000098a9e6f8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664318144:12345-0@lo:400:kworker/u8:0.0 Request processed in 27us (371us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189272.765333:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8491 00000100:00100000:1.0:1776189272.765369:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d4382e95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664317760:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189272.765377:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed846a61 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664318144:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189272.765384:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe93ff8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664317632:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189272.765391:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dabf4d62 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664318080:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189272.765865:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000050c951fb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664317696:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189272.765898:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189272.765919:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b864aa6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664317824:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189272.765937:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189272.765947:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004447f539 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664317888:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189272.765967:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189272.765997:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664317696 00000100:00100000:2.0:1776189272.766011:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664317824 00000100:00100000:2.0:1776189272.766018:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57302 00000100:00100000:2.0:1776189272.766022:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000be3c8080 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664317696:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189272.766036:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189272.766065:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000be3c8080 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664317696:12345-0@lo:38:kworker/u8:0.0 Request processed in 43us (167us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189272.766071:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57302 00000100:00100000:2.0:1776189272.766080:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57303 00000100:00100000:2.0:1776189272.766083:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e6117432 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664317824:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189272.766089:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189272.766114:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e6117432 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664317824:12345-0@lo:38:kworker/u8:0.0 Request processed in 21us (168us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189272.766120:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57303 00000100:00100000:2.0:1776189272.766144:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664317888 00000100:00100000:2.0:1776189272.766159:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9100 00000100:00100000:2.0:1776189272.766162:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000088ec3431 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664317888:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189272.766168:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189272.766208:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000088ec3431 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664317888:12345-0@lo:38:kworker/u8:0.0 Request processed in 44us (240us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189272.766214:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9100 00000100:00080000:2.0:1776189272.766238:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189272.766242:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189272.766245:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000050c951fb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664317696:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776189272.766252:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189272.766255:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189272.766256:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b864aa6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664317824:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189272.766449:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003258ba66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664317568:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189272.766538:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a943cb24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664317952:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189272.766648:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b827fed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664318336:0@lo:13:osp-pre-0-1.0 00000100:00080000:2.0:1776189272.766680:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189272.766684:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189272.766687:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004447f539 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664317888:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189272.767050:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189272.767077:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc4b762b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664318016:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189272.767099:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189272.767121:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000887d8a72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664318272:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189272.767139:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189272.767165:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664318272 00000100:00100000:3.0:1776189272.767222:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6706 00000100:00100000:3.0:1776189272.767227:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000024c95b46 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664318272:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189272.767245:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189272.767460:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664317568 00000100:00100000:2.0:1776189272.767488:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1411 00000100:00100000:2.0:1776189272.767492:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008da73e5e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664317568:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189272.767555:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000024c95b46 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664318272:12345-0@lo:13:osp-pre-1-1.0 Request processed in 327us (414us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189272.767562:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6706 00000100:00100000:3.0:1776189272.767803:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000887d8a72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664318272:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189272.767964:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189272.767993:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba2818a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664318208:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189272.768015:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189272.768086:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664318016 00000100:00100000:0.0:1776189272.768125:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664317952 00000100:00100000:0.0:1776189272.768142:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664318208 00000100:00100000:0.0:1776189272.768157:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8492 00000100:00100000:0.0:1776189272.768161:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003d6663c8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664318016:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189272.768215:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003d6663c8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664318016:12345-0@lo:400:kworker/u8:0.0 Request processed in 53us (1116us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189272.768221:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8492 00000100:00100000:0.0:1776189272.768233:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8493 00000100:00100000:0.0:1776189272.768237:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e0bab4ab pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664317952:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189272.768278:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e0bab4ab pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664317952:12345-0@lo:400:kworker/u8:0.0 Request processed in 40us (314us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189272.768283:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8493 00000100:00100000:0.0:1776189272.768293:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8494 00000100:00100000:0.0:1776189272.768297:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007a543fbe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664318208:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189272.768321:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007a543fbe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664318208:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (306us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189272.768327:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8494 00000100:00100000:0.0:1776189272.768358:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a943cb24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664317952:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189272.768367:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba2818a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664318208:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189272.768820:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189272.769119:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc4b762b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664318016:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189272.769257:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664318336 00000100:00100000:0.0:1776189272.769304:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6707 00000100:00100000:0.0:1776189272.769308:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f159c11f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664318336:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776189272.769327:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189272.769374:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f159c11f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664318336:12345-0@lo:13:osp-pre-0-1.0 Request processed in 65us (557us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189272.769386:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6707 00000100:00100000:1.0:1776189272.769459:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b827fed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664318336:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189272.770233:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003258ba66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664317568:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189272.770379:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008da73e5e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664317568:12345-0@lo:400:kworker/u8:0.0 Request processed in 2884us (3329us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189272.770388:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1411 00010000:02020000:2.0:1776189272.788284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189273.044292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189273.086754:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189273.300488:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189273.556288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189273.816149:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189274.068289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189274.108365:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189274.324285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189274.580279:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189274.836277:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189275.092283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189275.132450:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189275.348294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189275.604294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189275.860283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189276.116283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189276.156452:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189276.373204:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189276.538924:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189276.538986:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189276.539027:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654949504 00000100:00100000:1.0:1776189276.539090:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9868 00000100:00100000:1.0:1776189276.539097:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000035725ab pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654949504:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776189276.539214:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000035725ab pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654949504:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 116us (297us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189276.539221:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9868 00000100:00100000:3.0:1776189276.539236:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654949440 00000100:00100000:3.0:1776189276.539267:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1412 00000100:00100000:3.0:1776189276.539271:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c3d39af pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654949440:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1776189276.539297:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c3d39af pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654949440:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 25us (312us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189276.539303:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1412 00000100:00100000:0.0:1776189276.552871:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189276.552979:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654949568 00000100:00100000:1.0:1776189276.553025:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57304 00000100:00100000:1.0:1776189276.553031:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000011679d4d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654949568:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:1.0:1776189276.553051:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189276.553126:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000011679d4d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654949568:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 94us (263us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189276.553132:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57304 00010000:02020000:2.0:1776189276.633954:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189276.860543:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000008468a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664318400:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189276.860626:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189276.861745:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664318400 00000100:00100000:3.0:1776189276.861804:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6708 00000100:00100000:3.0:1776189276.861810:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000099b8eb0d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664318400:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776189276.861843:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189276.861908:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000099b8eb0d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664318400:12345-0@lo:13:osp-pre-1-0.0 Request processed in 97us (1282us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189276.861915:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6708 00000100:00100000:2.0:1776189276.862549:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000008468a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664318400:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189276.864892:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776189276.864913:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b827fed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664318464:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189276.864983:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189276.865338:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664318464 00000100:00100000:1.0:1776189276.865394:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6709 00000100:00100000:1.0:1776189276.865400:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000887d8a72 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664318464:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189276.865427:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189276.865485:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000887d8a72 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664318464:12345-0@lo:13:osp-pre-0-0.0 Request processed in 84us (502us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189276.865492:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6709 00000100:00100000:2.0:1776189276.865786:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b827fed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664318464:0@lo:13:osp-pre-0-0.0 00010000:02020000:2.0:1776189276.884297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189277.140284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189277.183261:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189277.396294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189277.652417:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189277.884527:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00080000:1.0:1776189277.884535:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189277.884541:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189277.884544:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1776189277.884550:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001b6b6588 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664318528:0@lo:13:osp-pre-1-1.0 00000100:00080000:1.0:1776189277.884551:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9419 00000100:00080000:1.0:1776189277.884556:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00080000:1.0:1776189277.884563:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189277.884581:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ced31f93 x1862459664318720/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189277.884602:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189277.884605:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189277.884607:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189277.884610:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9419 00000100:00080000:1.0:1776189277.884613:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00100000:3.0:1776189277.884614:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776189277.884619:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189277.884685:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000201d0f3c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664318848:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189277.884718:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ced31f93 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664318720:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189277.884759:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189277.884812:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d72c99d5 x1862459664318912/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189277.884843:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664318720 00000100:00100000:2.0:1776189277.884876:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57305 00000100:00100000:2.0:1776189277.884881:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000266f5a0c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664318720:12345-0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189277.884894:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664318528 00010000:02000000:2.0:1776189277.884897:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189277.884913:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a0fb2966 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664318784:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189277.884947:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189277.884999:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6710 00000100:00100000:3.0:1776189277.885002:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189277.885004:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba2818a5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664318528:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189277.885023:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189277.885038:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664318848 00000100:00100000:3.0:1776189277.885059:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6711 00000100:00100000:0.0:1776189277.885061:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba2818a5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664318528:12345-0@lo:13:osp-pre-1-1.0 Request processed in 57us (449us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189277.885067:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6710 00000100:00100000:3.0:1776189277.885074:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cf6b5d65 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664318848:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776189277.885085:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189277.885099:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000033264082 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664318592:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189277.885115:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cf6b5d65 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664318848:12345-0@lo:13:osp-pre-0-1.0 Request processed in 50us (112us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189277.885121:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6711 00000100:00100000:0.0:1776189277.885125:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189277.885140:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776189277.885144:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e39f5436 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664318656:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189277.885161:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001b6b6588 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664318528:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189277.885163:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189277.885209:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000201d0f3c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664318848:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189277.885232:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664318592 00000100:00100000:3.0:1776189277.885244:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664318784 00000100:00100000:0.0:1776189277.885255:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1413 00000100:00100000:0.0:1776189277.885258:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e858e44a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664318592:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189277.885263:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664318656 00000100:00100000:3.0:1776189277.885279:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9869 00000100:00100000:3.0:1776189277.885282:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007df90c3e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664318784:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189277.885289:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e858e44a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664318592:12345-0@lo:400:kworker/u8:0.0 Request processed in 30us (164us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189277.885295:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1413 00000100:00100000:3.0:1776189277.885312:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007df90c3e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664318784:12345-0@lo:400:kworker/u8:0.0 Request processed in 29us (365us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189277.885318:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9869 00000100:00100000:0.0:1776189277.885325:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000033264082 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664318592:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189277.885328:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9870 00000100:00100000:3.0:1776189277.885331:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d1c360e6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664318656:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189277.885364:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d1c360e6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664318656:12345-0@lo:400:kworker/u8:0.0 Request processed in 31us (200us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189277.885369:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9870 00000100:00100000:0.0:1776189277.885376:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e39f5436 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664318656:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189277.885388:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a0fb2966 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664318784:0@lo:400:kworker/u8:0.0 00000100:00080000:1.0:1776189277.890957:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189277.890976:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189277.890979:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189277.890988:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9419 00000100:00080000:1.0:1776189277.890992:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 6s 00000100:00080000:1.0:1776189277.890999:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189277.891000:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d72c99d5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664318912:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189277.891035:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000040e81793 x1862459664318976/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189277.891081:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189277.891127:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000266f5a0c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664318720:12345-0@lo:38:kworker/u8:0.0 Request processed in 6243us (6366us total) trans 0 rc -19/-19 00000100:00080000:3.0:1776189277.891128:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189277.891135:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189277.891140:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ced31f93 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664318720:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189277.891143:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57305 00000100:00100000:3.0:1776189277.891168:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000040e81793 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664318976:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189277.891169:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664318912 00000100:00100000:3.0:1776189277.891219:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189277.891258:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664318976 00000100:00100000:2.0:1776189277.891287:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9101 00000100:00100000:2.0:1776189277.891292:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f435eca pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664318976:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189277.891307:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189277.891347:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f435eca pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664318976:12345-0@lo:38:kworker/u8:0.0 Request processed in 54us (128us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189277.891353:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9101 00000100:00080000:2.0:1776189277.891376:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189277.891379:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189277.891382:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000040e81793 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664318976:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189277.891606:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57306 00000100:00100000:0.0:1776189277.891610:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004dd53b72 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664318912:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189277.891619:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189277.893538:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004dd53b72 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664318912:12345-0@lo:38:kworker/u8:0.0 Request processed in 1920us (2454us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189277.893555:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57306 00000100:00100000:1.0:1776189277.893705:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a0fb2966 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664319168:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189277.893753:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189277.893798:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664319168 00000100:00100000:1.0:1776189277.893834:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9871 00000100:00100000:1.0:1776189277.893840:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e39f5436 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664319168:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189277.893885:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e39f5436 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664319168:12345-0@lo:400:kworker/u8:0.0 Request processed in 45us (132us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189277.893892:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9871 00000100:00100000:1.0:1776189277.893928:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a0fb2966 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664319168:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189277.894009:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000040e81793 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664319040:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189277.894049:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189277.894078:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776189277.894084:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ced31f93 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664319104:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189277.894103:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189277.894129:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664319040 00000100:00100000:0.0:1776189277.894152:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664319104 00000100:00100000:0.0:1776189277.894169:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8495 00000100:00100000:0.0:1776189277.894200:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ae28b57d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664319040:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189277.894230:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8496 00000100:00100000:3.0:1776189277.894237:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000008468a4e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664319104:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189277.894243:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ae28b57d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664319040:12345-0@lo:400:kworker/u8:0.0 Request processed in 41us (192us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189277.894251:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8495 00000100:00100000:3.0:1776189277.894276:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000008468a4e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664319104:12345-0@lo:400:kworker/u8:0.0 Request processed in 39us (172us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189277.894283:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8496 00000100:00100000:0.0:1776189277.894289:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ced31f93 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664319104:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189277.894304:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000040e81793 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664319040:0@lo:400:kworker/u8:0.0 00000100:00080000:2.0:1776189277.897562:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189277.897572:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189277.897579:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d72c99d5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664318912:0@lo:38:kworker/u8:0.0 00010000:02020000:2.0:1776189277.908284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189278.164281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189278.205484:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189278.420534:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189278.676290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189278.932276:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189279.188348:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189279.229013:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189279.444306:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189279.700378:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189279.956328:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189280.212283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189280.252476:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189280.471014:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189280.724327:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189280.980275:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189281.236320:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189281.276622:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189281.492292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189281.660403:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189281.660510:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189281.660549:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189281.660608:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654949632 00000100:00100000:0.0:1776189281.660666:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1414 00000100:00100000:0.0:1776189281.660673:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ff8d0a76 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654949632:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189281.660758:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ff8d0a76 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654949632:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 70us (233us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189281.660766:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1414 00000100:00100000:1.0:1776189281.662121:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654949696 00000100:00100000:1.0:1776189281.662212:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9872 00000100:00100000:1.0:1776189281.662218:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d9b9c94e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654949696:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189281.662280:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d9b9c94e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654949696:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 62us (1886us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189281.662287:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9872 00000100:00100000:1.0:1776189281.662305:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654949760 00000100:00100000:0.0:1776189281.662346:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57307 00000100:00100000:0.0:1776189281.662361:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ace99b9b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654949760:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776189281.662376:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189281.662403:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ace99b9b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654949760:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 51us (1855us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189281.662409:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57307 00010000:02020000:2.0:1776189281.748290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189281.980489:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776189281.980511:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000089232900 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664319232:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189281.980579:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189281.980888:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a0fb2966 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664319296:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189281.980917:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189281.980999:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664319232 00000100:00100000:0.0:1776189281.981046:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664319296 00000100:00100000:0.0:1776189281.981070:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6712 00000100:00100000:0.0:1776189281.981074:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000201d0f3c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664319232:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189281.981103:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189281.981144:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000201d0f3c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664319232:12345-0@lo:13:osp-pre-1-0.0 Request processed in 68us (565us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189281.981152:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6712 00000100:00100000:0.0:1776189281.981202:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6713 00000100:00100000:0.0:1776189281.981206:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000050c951fb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664319296:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776189281.981222:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189281.981250:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000050c951fb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664319296:12345-0@lo:13:osp-pre-0-0.0 Request processed in 43us (333us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189281.981256:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6713 00000100:00100000:2.0:1776189281.981492:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a0fb2966 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664319296:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189281.981524:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000089232900 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664319232:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189282.004329:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189282.261914:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189282.300700:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189282.518474:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189282.772362:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189283.004626:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776189283.004948:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001b6b6588 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664319424:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189283.005013:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189283.005111:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664319424 00000100:00100000:2.0:1776189283.005161:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6714 00000100:00100000:2.0:1776189283.005167:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c7c1f715 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664319424:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189283.005224:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189283.005267:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c7c1f715 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664319424:12345-0@lo:13:osp-pre-1-1.0 Request processed in 100us (254us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189283.005275:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6714 00000100:00100000:3.0:1776189283.005500:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001b6b6588 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664319424:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189283.005735:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004d99e80b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664319360:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189283.005768:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189283.005953:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664319360 00000100:00100000:3.0:1776189283.005984:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6715 00000100:00100000:3.0:1776189283.005989:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e9c407a2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664319360:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776189283.006003:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189283.006082:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776189283.006094:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000033264082 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664319488:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189283.006416:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e9c407a2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664319360:12345-0@lo:13:osp-pre-0-1.0 Request processed in 425us (647us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189283.006424:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6715 00000100:00100000:3.0:1776189283.006464:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b827fed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664319552:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189283.006533:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189283.006575:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664319488 00000100:00100000:2.0:1776189283.006602:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1415 00000100:00100000:2.0:1776189283.006606:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f07038bd pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664319488:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189283.007108:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189283.007161:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664319552 00000100:00100000:3.0:1776189283.007222:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9873 00000100:00100000:3.0:1776189283.007226:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001b6b6588 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664319552:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189283.007268:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001b6b6588 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664319552:12345-0@lo:400:kworker/u8:1.0 Request processed in 40us (161us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189283.007274:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9873 00000100:00100000:2.0:1776189283.008460:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f07038bd pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664319488:12345-0@lo:400:kworker/u8:1.0 Request processed in 1847us (1922us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189283.008472:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1415 00000100:00100000:2.0:1776189283.008533:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000033264082 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664319488:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189283.009145:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b827fed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664319552:0@lo:400:kworker/u8:1.0 00000100:00080000:1.0:1776189283.012574:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189283.012584:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189283.012588:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189283.012596:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9424 00000100:00080000:1.0:1776189283.012600:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776189283.012605:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189283.012645:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009b827fed x1862459664319616/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189283.012703:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189283.012707:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189283.012708:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189283.012711:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9424 00000100:00080000:1.0:1776189283.012713:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776189283.012717:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189283.012726:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a0fb2966 x1862459664319744/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189283.012734:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189283.012736:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189283.012737:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189283.012740:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9424 00000100:00080000:1.0:1776189283.012742:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776189283.012744:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189283.012753:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d72c99d5 x1862459664319808/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189283.012825:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000040e81793 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664319872:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189283.013027:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004d99e80b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664319360:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189283.013057:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000033264082 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664319680:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189283.013151:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b827fed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664319616:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189283.015218:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ced31f93 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664319936:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189283.023189:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189283.023247:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776189283.023258:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003258ba66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664320000:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189283.023289:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189283.023368:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664319680 00000100:00100000:0.0:1776189283.023413:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664320000 00000100:00100000:0.0:1776189283.023432:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9874 00000100:00100000:0.0:1776189283.023436:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000031b5a624 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664319680:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189283.023500:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000031b5a624 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664319680:12345-0@lo:400:kworker/u8:1.0 Request processed in 63us (331us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189283.023508:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9874 00000100:00100000:0.0:1776189283.023525:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9875 00000100:00100000:0.0:1776189283.023528:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000412a95e8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664320000:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189283.023555:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000412a95e8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664320000:12345-0@lo:400:kworker/u8:1.0 Request processed in 26us (266us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189283.023561:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9875 00000100:00100000:0.0:1776189283.023601:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003258ba66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664320000:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189283.023612:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000033264082 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664319680:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189283.028576:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189283.028716:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664319872 00000100:00100000:0.0:1776189283.028774:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8497 00000100:00100000:0.0:1776189283.028780:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fe93ff8e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664319872:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189283.028845:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fe93ff8e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664319872:12345-0@lo:400:kworker/u8:1.0 Request processed in 64us (275us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189283.028862:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8497 00000100:00100000:2.0:1776189283.029031:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189283.029056:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a0fb2966 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664319744:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189283.029083:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189283.029097:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d72c99d5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664319808:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189283.029120:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189283.029156:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664319616 00000100:00100000:2.0:1776189283.029202:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664319744 00000100:00100000:2.0:1776189283.029218:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57308 00000100:00100000:2.0:1776189283.029222:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004d99e80b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664319616:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189283.029236:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189283.029267:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004d99e80b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664319616:12345-0@lo:38:kworker/u8:1.0 Request processed in 44us (237us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189283.029273:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57308 00000100:00100000:2.0:1776189283.029286:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57309 00000100:00100000:2.0:1776189283.029289:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cae4b12b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664319744:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189283.029295:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189283.029316:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cae4b12b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664319744:12345-0@lo:38:kworker/u8:1.0 Request processed in 26us (233us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189283.029332:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57309 00000100:00080000:2.0:1776189283.029372:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189283.029375:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189283.029379:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b827fed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664319616:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776189283.029387:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189283.029389:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189283.029391:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a0fb2966 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664319744:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189283.029430:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664319808 00000100:00100000:2.0:1776189283.029461:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9102 00000100:00100000:2.0:1776189283.029464:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000039f4c6c3 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664319808:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189283.029472:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189283.029500:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000039f4c6c3 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664319808:12345-0@lo:38:kworker/u8:1.0 Request processed in 35us (381us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189283.029506:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9102 00000100:00080000:2.0:1776189283.029527:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189283.029530:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189283.029533:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d72c99d5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664319808:0@lo:38:kworker/u8:1.0 00010000:02020000:0.0:1776189283.029705:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189283.030140:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189283.030300:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664319936 00000100:00100000:1.0:1776189283.030348:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8498 00000100:00100000:1.0:1776189283.030352:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b1d7bdea pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664319936:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189283.030401:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b1d7bdea pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664319936:12345-0@lo:400:kworker/u8:1.0 Request processed in 47us (261us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189283.030407:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8498 00000100:00100000:1.0:1776189283.030452:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000040e81793 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664319872:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189283.030653:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ced31f93 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664319936:0@lo:400:kworker/u8:1.0 00010000:02020000:0.0:1776189283.284400:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189283.324636:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189283.540307:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189283.796319:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189284.052362:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189284.308360:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189284.350334:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189284.564308:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189284.820308:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189285.077992:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189285.332283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189285.372490:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189285.588294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189285.844305:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189286.100759:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189286.356454:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189286.396359:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189286.612287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189286.779762:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189286.779863:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189286.779950:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189286.780016:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654949824 00000100:00100000:1.0:1776189286.780076:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1416 00000100:00100000:1.0:1776189286.780083:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005039a00d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654949824:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776189286.780158:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005039a00d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654949824:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 74us (295us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189286.780165:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1416 00000100:00100000:2.0:1776189286.787549:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654949952 00000100:00100000:2.0:1776189286.787618:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654949888 00000100:00100000:2.0:1776189286.787872:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9876 00000100:00100000:2.0:1776189286.787879:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000074a40efb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654949888:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776189286.787959:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000074a40efb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654949888:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 79us (8010us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189286.787966:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9876 00000100:00100000:2.0:1776189286.787986:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57310 00000100:00100000:2.0:1776189286.787989:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d9eb44b4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654949952:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:2.0:1776189286.788004:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189286.788026:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d9eb44b4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654949952:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 36us (8273us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189286.788032:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57310 00010000:02020000:0.0:1776189286.873376:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189287.100910:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776189287.100933:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003258ba66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664320064:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189287.101001:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189287.101106:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664320064 00000100:00100000:1.0:1776189287.101156:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6716 00000100:00100000:1.0:1776189287.101162:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e463b423 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664320064:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189287.101214:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189287.101271:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e463b423 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664320064:12345-0@lo:13:osp-pre-0-0.0 Request processed in 109us (271us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189287.101279:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6716 00000100:00100000:1.0:1776189287.101328:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003258ba66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664320064:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189287.102273:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776189287.102288:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002ed2a919 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664320128:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189287.102328:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189287.102542:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664320128 00000100:00100000:2.0:1776189287.102582:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6717 00000100:00100000:2.0:1776189287.102587:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000079c74b68 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664320128:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189287.102609:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189287.102651:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000079c74b68 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664320128:12345-0@lo:13:osp-pre-1-0.0 Request processed in 63us (323us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189287.102658:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6717 00000100:00100000:3.0:1776189287.102716:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002ed2a919 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664320128:0@lo:13:osp-pre-1-0.0 00010000:02020000:0.0:1776189287.124396:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189287.380915:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189287.420770:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189287.636306:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189287.892567:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189288.124357:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776189288.124378:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005f6664e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664320192:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189288.124442:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189288.124500:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664320192 00000100:00100000:3.0:1776189288.124541:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6718 00000100:00100000:3.0:1776189288.124547:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003258ba66 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664320192:12345-0@lo:13:osp-pre-1-1.0 00000100:00080000:2.0:1776189288.124574:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189288.124579:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00002000:00000024:3.0:1776189288.124581:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00080000:2.0:1776189288.124583:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189288.124590:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9429 00000100:00080000:2.0:1776189288.124594:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:2.0:1776189288.124599:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189288.124614:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001b0d11fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664320256:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189288.124623:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000381790a0 x1862459664320384/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189288.124631:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003258ba66 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664320192:12345-0@lo:13:osp-pre-1-1.0 Request processed in 81us (187us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189288.124642:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6718 00000100:00080000:2.0:1776189288.124662:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189288.124666:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1776189288.124669:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1776189288.124669:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189288.124675:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9429 00000100:00080000:2.0:1776189288.124677:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00080000:2.0:1776189288.124683:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189288.124683:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b5045f9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664320320:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189288.124700:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000032f89d8f x1862459664320512/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189288.124706:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664320256 00000100:00080000:2.0:1776189288.124710:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189288.124714:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189288.124716:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189288.124720:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9429 00000100:00080000:2.0:1776189288.124722:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00100000:3.0:1776189288.124726:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1776189288.124729:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189288.124740:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1417 00000100:00100000:2.0:1776189288.124745:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002fbab827 x1862459664320576/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189288.124747:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000080a5522a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664320256:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189288.124774:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000381790a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664320384:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189288.124819:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189288.124839:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000032f89d8f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664320512:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189288.124862:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007c850b41 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664320640:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189288.124873:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189288.124896:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002fbab827 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664320576:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189288.124902:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189288.124929:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189288.124931:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776189288.124939:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009be3a12f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664320448:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189288.124964:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664320320 00000100:00100000:0.0:1776189288.124968:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189288.125004:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664320384 00000100:00100000:0.0:1776189288.125032:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664320512 00000100:00100000:0.0:1776189288.125040:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664320448 00000100:00100000:0.0:1776189288.125053:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9877 00000100:00100000:0.0:1776189288.125057:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e5ef59b0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664320320:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189288.125088:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e5ef59b0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664320320:12345-0@lo:400:kworker/u8:0.0 Request processed in 30us (366us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189288.125093:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9878 00000100:00100000:0.0:1776189288.125100:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9877 00000100:00100000:3.0:1776189288.125102:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001706ae06 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664320448:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189288.125119:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57311 00000100:00100000:0.0:1776189288.125124:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003a715368 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664320384:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189288.125137:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189288.125146:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001706ae06 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664320448:12345-0@lo:400:kworker/u8:0.0 Request processed in 45us (179us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189288.125154:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9878 00000100:00100000:3.0:1776189288.125194:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57312 00000100:00100000:0.0:1776189288.125198:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003a715368 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664320384:12345-0@lo:38:kworker/u8:0.0 Request processed in 72us (379us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189288.125202:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000051b336a8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664320512:12345-0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189288.125210:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57311 00010000:02000000:3.0:1776189288.125224:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189288.125249:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664320640 00000100:00100000:3.0:1776189288.125254:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000051b336a8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664320512:12345-0@lo:38:kworker/u8:0.0 Request processed in 54us (381us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189288.125261:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57312 00000100:00100000:0.0:1776189288.125279:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8499 00000100:00100000:0.0:1776189288.125284:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000733a9e44 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664320640:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189288.125302:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009806e292 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664320768:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189288.125331:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000733a9e44 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664320640:12345-0@lo:400:kworker/u8:0.0 Request processed in 46us (430us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189288.125338:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189288.125345:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8499 00000100:00100000:3.0:1776189288.125374:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b5045f9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664320320:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189288.125385:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009be3a12f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664320448:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189288.125396:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776189288.125402:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007c850b41 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664320640:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189288.125405:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ada75b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664320704:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189288.125411:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000080a5522a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664320256:12345-0@lo:400:kworker/u8:0.0 Request processed in 662us (741us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189288.125421:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1417 00000100:00100000:0.0:1776189288.125436:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664320768 00000100:00100000:3.0:1776189288.125436:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189288.125472:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9879 00000100:00100000:1.0:1776189288.125479:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000061a1fd5c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664320768:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189288.125492:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664320704 00000100:00100000:3.0:1776189288.125516:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664320576 00000100:00100000:0.0:1776189288.125520:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8500 00000100:00100000:1.0:1776189288.125526:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000061a1fd5c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664320768:12345-0@lo:400:kworker/u8:0.0 Request processed in 46us (191us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189288.125527:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009b5045f9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664320704:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189288.125539:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9879 00000100:00100000:3.0:1776189288.125544:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9103 00000100:00100000:3.0:1776189288.125550:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004ae527b6 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664320576:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189288.125559:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189288.125569:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001b0d11fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664320256:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189288.125591:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004ae527b6 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664320576:12345-0@lo:38:kworker/u8:0.0 Request processed in 40us (663us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189288.125597:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d0281336 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664320832:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189288.125600:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9103 00000100:00100000:1.0:1776189288.125632:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189288.125634:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009806e292 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664320768:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189288.125660:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664320832 00000100:00100000:1.0:1776189288.125668:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f6664e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664320192:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189288.125684:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6719 00000100:00100000:3.0:1776189288.125690:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001b0d11fc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664320832:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776189288.125701:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00080000:1.0:1776189288.125718:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189288.125723:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189288.125727:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000381790a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664320384:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776189288.125739:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189288.125743:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189288.125743:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001b0d11fc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664320832:12345-0@lo:13:osp-pre-0-1.0 Request processed in 51us (111us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189288.125750:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000032f89d8f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664320512:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189288.125754:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6719 00000100:00080000:1.0:1776189288.125765:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189288.125768:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189288.125772:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002fbab827 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664320576:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189288.125808:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d0281336 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664320832:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189288.126783:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009b5045f9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664320704:12345-0@lo:400:kworker/u8:0.0 Request processed in 1247us (1339us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189288.126791:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8500 00000100:00100000:3.0:1776189288.127480:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ada75b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664320704:0@lo:400:kworker/u8:0.0 00010000:02020000:0.0:1776189288.149111:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189288.404291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189288.444388:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189288.660816:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189288.916698:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189289.176755:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189289.428297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189289.468458:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189289.684493:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189289.940548:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189290.196417:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189290.454894:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189290.492595:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189290.708286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189290.964252:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189291.220388:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189291.477879:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189291.516626:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189291.732723:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189291.898499:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189291.898541:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189291.899007:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654950016 00000100:00100000:3.0:1776189291.899126:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189291.899160:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1418 00000100:00100000:2.0:1776189291.899168:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f99b7a85 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654950016:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776189291.899205:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654950144 00000100:00100000:1.0:1776189291.899236:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654950080 00000100:00100000:1.0:1776189291.899260:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9880 00000100:00100000:1.0:1776189291.899265:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000869e3fc2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654950080:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1776189291.899314:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57313 00000100:00100000:3.0:1776189291.899322:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000005562bb2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654950144:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00000100:00100000:1.0:1776189291.899331:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000869e3fc2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654950080:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 64us (206us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189291.899341:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9880 00010000:02000000:3.0:1776189291.899346:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776189291.899389:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000005562bb2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654950144:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 67us (847us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189291.899399:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57313 00000100:00100000:2.0:1776189291.900055:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f99b7a85 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654950016:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 885us (1562us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189291.900064:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1418 00010000:02020000:2.0:1776189291.988914:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189292.220537:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002fbab827 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664320896:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189292.220585:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776189292.220600:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d8f74173 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664320960:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189292.220616:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189292.220678:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664320896 00000100:00100000:3.0:1776189292.220746:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6720 00000100:00100000:3.0:1776189292.220751:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000af79f780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664320896:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189292.220782:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189292.221014:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189292.222551:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664320960 00000100:00100000:1.0:1776189292.222701:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6721 00000100:00100000:1.0:1776189292.222708:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a5008497 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664320960:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189292.222738:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189292.222803:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a5008497 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664320960:12345-0@lo:13:osp-pre-1-0.0 Request processed in 95us (1790us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189292.223142:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d8f74173 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664320960:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189292.225836:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000af79f780 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664320896:12345-0@lo:13:osp-pre-0-0.0 Request processed in 5075us (5214us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189292.225852:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6720 00000100:00100000:1.0:1776189292.229225:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6721 00000100:00100000:0.0:1776189292.230829:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002fbab827 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664320896:0@lo:13:osp-pre-0-0.0 00010000:02020000:2.0:1776189292.244344:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189292.500441:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189292.540579:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189292.756288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189293.012310:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189293.244397:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002fbab827 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664321024:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189293.244423:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776189293.244435:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a9575a34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664321088:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189293.244468:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189293.244489:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664321088 00000100:00100000:2.0:1776189293.244502:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189293.244522:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664321024 00000100:00100000:2.0:1776189293.244529:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6722 00000100:00100000:2.0:1776189293.244535:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000917ca813 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664321088:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189293.244559:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189293.244594:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000917ca813 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664321088:12345-0@lo:13:osp-pre-0-1.0 Request processed in 58us (126us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189293.246145:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6722 00000100:00100000:0.0:1776189293.246165:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6723 00000100:00100000:0.0:1776189293.246191:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001ada75b0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664321024:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189293.246214:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189293.246236:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a9575a34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664321088:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189293.246284:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001ada75b0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664321024:12345-0@lo:13:osp-pre-1-1.0 Request processed in 110us (1780us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189293.246292:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6723 00000100:00100000:2.0:1776189293.246303:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002fbab827 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664321024:0@lo:13:osp-pre-1-1.0 00000100:00080000:1.0:1776189293.248057:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189293.248063:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189293.248067:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1776189293.248070:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00080000:1.0:1776189293.248075:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9435 00000100:00080000:1.0:1776189293.248080:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189293.248085:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189293.248089:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002fbab827 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664321152:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189293.248117:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005f6664e0 x1862459664321280/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189293.248154:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189293.248158:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189293.248161:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1776189293.248162:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776189293.248166:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9435 00000100:00080000:1.0:1776189293.248170:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189293.248204:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189293.248217:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ed846a61 x1862459664321408/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189293.248224:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1776189293.248225:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664321152 00000100:00080000:1.0:1776189293.248229:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189293.248231:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189293.248234:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9435 00000100:00080000:1.0:1776189293.248236:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189293.248239:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189293.248251:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cc4b762b x1862459664321472/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189293.248269:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1419 00000100:00100000:0.0:1776189293.248275:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000afb92d61 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664321152:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189293.248320:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000afb92d61 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664321152:12345-0@lo:400:kworker/u8:0.0 Request processed in 44us (160us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189293.248328:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1419 00000100:00100000:0.0:1776189293.248370:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005f6664e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664321280:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189293.248397:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189293.248413:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed846a61 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664321408:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189293.248434:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189293.248450:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc4b762b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664321472:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189293.248469:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a943cb24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664321536:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189293.248475:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189293.248521:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189293.248523:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664321280 00000100:00100000:0.0:1776189293.248549:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664321408 00000100:00100000:1.0:1776189293.248562:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002fbab827 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664321152:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189293.248567:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57314 00000100:00100000:0.0:1776189293.248572:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000895d417a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664321280:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189293.248586:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189293.248586:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776189293.248594:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004447f539 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664321600:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189293.248630:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000895d417a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664321280:12345-0@lo:38:kworker/u8:0.0 Request processed in 56us (231us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189293.248637:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57314 00000100:00100000:1.0:1776189293.248648:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189293.248665:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57315 00000100:00100000:0.0:1776189293.248669:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000e15ac1c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664321408:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189293.248677:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189293.248684:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664321536 00000100:00100000:1.0:1776189293.248717:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664321600 00000100:00100000:1.0:1776189293.248740:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8501 00000100:00100000:1.0:1776189293.248744:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d4382e95 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664321536:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189293.248789:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d4382e95 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664321536:12345-0@lo:400:kworker/u8:0.0 Request processed in 38us (271us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189293.248796:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8501 00000100:00100000:1.0:1776189293.248808:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8502 00000100:00100000:1.0:1776189293.248811:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002fbab827 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664321600:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189293.248845:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002fbab827 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664321600:12345-0@lo:400:kworker/u8:0.0 Request processed in 33us (212us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189293.248859:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8502 00000100:00100000:1.0:1776189293.248901:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004447f539 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664321600:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189293.248909:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a943cb24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664321536:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189293.248913:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000e15ac1c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664321408:12345-0@lo:38:kworker/u8:0.0 Request processed in 35us (272us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189293.248954:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000032f89d8f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664321216:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189293.248984:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189293.248996:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b864aa6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664321664:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189293.249020:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189293.249024:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57315 00000100:00100000:1.0:1776189293.249035:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776189293.249040:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664321216 00000100:00100000:1.0:1776189293.249043:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dabf4d62 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664321344:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189293.249067:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189293.249070:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664321664 00000100:00100000:0.0:1776189293.249086:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664321344 00000100:00100000:0.0:1776189293.249112:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9881 00000100:00100000:0.0:1776189293.249116:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a943cb24 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664321216:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189293.249128:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9882 00000100:00100000:1.0:1776189293.249134:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004447f539 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664321664:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189293.249157:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a943cb24 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664321216:12345-0@lo:400:kworker/u8:0.0 Request processed in 39us (172us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189293.249164:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9881 00000100:00100000:0.0:1776189293.249202:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9883 00000100:00100000:0.0:1776189293.249209:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001929a2c4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664321344:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189293.249210:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004447f539 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664321664:12345-0@lo:400:kworker/u8:0.0 Request processed in 73us (187us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189293.249221:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9882 00000100:00100000:0.0:1776189293.249247:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001929a2c4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664321344:12345-0@lo:400:kworker/u8:0.0 Request processed in 39us (181us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189293.249255:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dabf4d62 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664321344:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189293.249256:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9883 00000100:00100000:1.0:1776189293.249272:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000032f89d8f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664321216:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189293.249283:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b864aa6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664321664:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189293.249286:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664321472 00000100:00100000:0.0:1776189293.249311:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9104 00000100:00100000:0.0:1776189293.249315:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002ed2a919 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664321472:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189293.249324:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189293.250691:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002ed2a919 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664321472:12345-0@lo:38:kworker/u8:0.0 Request processed in 1371us (2212us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189293.250702:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9104 00000100:00080000:1.0:1776189293.252204:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189293.252212:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189293.252220:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f6664e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664321280:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776189293.252248:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189293.252251:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189293.252253:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed846a61 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664321408:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776189293.252263:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189293.252265:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189293.252268:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc4b762b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664321472:0@lo:38:kworker/u8:0.0 00010000:02020000:2.0:1776189293.268353:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189293.524355:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189293.565191:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189293.780283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189294.036792:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189294.292332:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189294.548247:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189294.588362:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189294.804351:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189295.060289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189295.316291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189295.572405:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189295.612571:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189295.828334:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189296.084343:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189296.340301:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189296.596288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189296.636483:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189296.852276:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189297.020004:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189297.020103:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189297.020224:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189297.020295:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654950208 00000100:00100000:3.0:1776189297.020348:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1420 00000100:00100000:3.0:1776189297.020354:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008f85e8f7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654950208:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189297.020427:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008f85e8f7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654950208:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 73us (204us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189297.020435:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1420 00000100:00100000:0.0:1776189297.020800:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654950336 00000100:00100000:0.0:1776189297.020834:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654950272 00000100:00100000:0.0:1776189297.020862:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9884 00000100:00100000:0.0:1776189297.020867:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009806e292 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654950272:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189297.020899:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009806e292 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654950272:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 31us (797us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189297.020905:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9884 00000100:00100000:0.0:1776189297.020918:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57316 00000100:00100000:0.0:1776189297.020922:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a9575a34 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654950336:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776189297.020936:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189297.020958:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a9575a34 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654950336:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 35us (963us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189297.020964:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57316 00010000:02020000:2.0:1776189297.108281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189297.340817:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776189297.340840:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005f6664e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664321728:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189297.340903:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189297.341024:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664321728 00000100:00100000:0.0:1776189297.341077:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6724 00000100:00100000:0.0:1776189297.341082:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006be5f924 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664321728:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776189297.341111:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189297.341161:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006be5f924 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664321728:12345-0@lo:13:osp-pre-0-0.0 Request processed in 78us (259us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189297.341167:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6724 00000100:00100000:3.0:1776189297.341326:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f6664e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664321728:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189297.341446:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005f6664e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664321792:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189297.341477:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189297.342054:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664321792 00000100:00100000:0.0:1776189297.342093:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6725 00000100:00100000:0.0:1776189297.342098:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f326e309 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664321792:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189297.342117:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189297.345924:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f326e309 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664321792:12345-0@lo:13:osp-pre-1-0.0 Request processed in 3818us (4438us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189297.345944:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6725 00000100:00100000:2.0:1776189297.346057:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f6664e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664321792:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189297.368081:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189297.623859:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189297.660486:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189297.876809:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189298.132294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:1.0:1776189298.364378:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189298.364386:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189298.364388:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189298.364396:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9440 00000100:00080000:1.0:1776189298.364400:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1776189298.364400:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00080000:1.0:1776189298.364405:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189298.364418:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc4b762b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664321920:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189298.364434:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009b864aa6 x1862459664321984/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189298.364454:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189298.364457:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189298.364458:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189298.364461:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9440 00000100:00080000:1.0:1776189298.364463:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189298.364465:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189298.364473:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ed846a61 x1862459664322112/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189298.364480:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189298.364482:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189298.364483:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189298.364486:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9440 00000100:00080000:1.0:1776189298.364488:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189298.364490:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189298.364498:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000dabf4d62 x1862459664322176/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189298.364502:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189298.364535:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000032f89d8f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664322048:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189298.364557:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189298.364562:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b864aa6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664321984:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189298.364577:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664321920 00000100:00100000:2.0:1776189298.364577:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a37ce050 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664322432:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189298.364606:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189298.364612:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189298.364655:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776189298.364661:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000932debd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664322368:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189298.364738:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005f6664e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664321856:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189298.364771:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189298.364787:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b54d6c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664322304:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189298.364806:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189298.364820:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c5b17f98 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664322560:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189298.364851:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189298.364864:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1776189298.364869:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a829435f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664322240:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189298.364886:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189298.364897:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005929d549 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664322496:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189298.364914:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189298.364936:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664321856 00000100:00100000:3.0:1776189298.364960:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1421 00000100:00100000:3.0:1776189298.364964:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000034f05964 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664321856:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189298.365017:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000034f05964 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664321856:12345-0@lo:400:kworker/u8:0.0 Request processed in 52us (246us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189298.365024:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1421 00000100:00100000:3.0:1776189298.365064:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f6664e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664321856:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189298.365108:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664322432 00000100:00100000:3.0:1776189298.365131:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664322304 00000100:00100000:3.0:1776189298.365145:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664322240 00000100:00100000:3.0:1776189298.365157:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664322496 00000100:00100000:3.0:1776189298.365190:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8503 00000100:00100000:3.0:1776189298.365194:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0146fc7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664322432:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189298.365231:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0146fc7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664322432:12345-0@lo:400:kworker/u8:0.0 Request processed in 35us (620us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189298.365236:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8503 00000100:00100000:3.0:1776189298.365247:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8504 00000100:00100000:3.0:1776189298.365250:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000607ae435 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664322304:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189298.365276:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000607ae435 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664322304:12345-0@lo:400:kworker/u8:0.0 Request processed in 26us (470us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189298.365281:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8504 00000100:00100000:3.0:1776189298.365290:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8505 00000100:00100000:3.0:1776189298.365293:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001065b146 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664322240:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189298.365315:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001065b146 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664322240:12345-0@lo:400:kworker/u8:0.0 Request processed in 21us (429us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189298.365320:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8505 00000100:00100000:3.0:1776189298.365329:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8506 00000100:00100000:3.0:1776189298.365332:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000624c000a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664322496:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189298.365353:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000624c000a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664322496:12345-0@lo:400:kworker/u8:0.0 Request processed in 20us (438us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189298.365357:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8506 00000100:00100000:3.0:1776189298.365396:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664322560 00000100:00100000:3.0:1776189298.365414:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6726 00000100:00100000:3.0:1776189298.365417:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002c38fc42 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664322560:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776189298.365429:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189298.365455:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002c38fc42 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664322560:12345-0@lo:13:osp-pre-0-1.0 Request processed in 37us (603us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189298.365461:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6726 00000100:00100000:3.0:1776189298.365476:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1b59934 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664322624:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189298.365498:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189298.365521:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664322624 00000100:00100000:3.0:1776189298.365539:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6727 00000100:00100000:3.0:1776189298.365542:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a4832ec7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664322624:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189298.365550:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189298.365577:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a4832ec7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664322624:12345-0@lo:13:osp-pre-1-1.0 Request processed in 34us (80us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189298.365582:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6727 00000100:00100000:3.0:1776189298.365608:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1b59934 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664322624:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189298.365631:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a829435f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664322240:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189298.365639:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005929d549 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664322496:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189298.365646:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b54d6c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664322304:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189298.365657:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c5b17f98 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664322560:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189298.367080:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed846a61 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664322112:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189298.367132:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189298.368336:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664322048 00000100:00100000:1.0:1776189298.368401:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664321984 00000100:00100000:1.0:1776189298.368599:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9885 00000100:00100000:1.0:1776189298.368607:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000031aca57e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664321920:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189298.368697:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000031aca57e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664321920:12345-0@lo:400:kworker/u8:0.0 Request processed in 90us (4195us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189298.368703:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9885 00000100:00100000:1.0:1776189298.368721:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9886 00000100:00100000:1.0:1776189298.368725:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a50d0fa6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664322048:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189298.368750:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a50d0fa6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664322048:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (4192us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189298.368755:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9886 00000100:00100000:1.0:1776189298.368764:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57317 00000100:00100000:1.0:1776189298.368766:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000089232900 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664321984:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189298.368781:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189298.368807:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000089232900 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664321984:12345-0@lo:38:kworker/u8:0.0 Request processed in 39us (4204us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189298.368812:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57317 00000100:00100000:0.0:1776189298.370603:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dabf4d62 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664322176:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189298.370659:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776189298.370711:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189298.370715:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189298.370720:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b864aa6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664321984:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189298.370733:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664322176 00000100:00100000:2.0:1776189298.370770:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9105 00000100:00100000:2.0:1776189298.370776:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000381790a0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664322176:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189298.370792:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189298.370839:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000381790a0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664322176:12345-0@lo:38:kworker/u8:0.0 Request processed in 63us (181us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189298.370845:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9105 00000100:00100000:3.0:1776189298.371169:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664322112 00000100:00100000:3.0:1776189298.371224:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57318 00000100:00100000:3.0:1776189298.371228:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007c850b41 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664322112:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189298.371236:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189298.371266:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007c850b41 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664322112:12345-0@lo:38:kworker/u8:0.0 Request processed in 37us (4135us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189298.371273:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57318 00000100:00100000:1.0:1776189298.371342:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189298.371379:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a37ce050 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664322432:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189298.371391:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000032f89d8f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664322048:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189298.371401:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc4b762b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664321920:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189298.371533:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664322368 00000100:00100000:2.0:1776189298.371567:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9887 00000100:00100000:2.0:1776189298.371571:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007ce83f84 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664322368:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189298.371614:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007ce83f84 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664322368:12345-0@lo:400:kworker/u8:0.0 Request processed in 43us (273us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189298.371621:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9887 00000100:00100000:2.0:1776189298.371652:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000932debd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664322368:0@lo:400:kworker/u8:0.0 00000100:00080000:0.0:1776189298.372134:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189298.372139:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189298.372143:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed846a61 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664322112:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776189298.372153:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189298.372155:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189298.372158:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dabf4d62 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664322176:0@lo:38:kworker/u8:0.0 00010000:02020000:2.0:1776189298.388794:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189298.644678:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189298.684582:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189298.903104:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189299.156289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189299.412638:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189299.668281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189299.709094:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189299.924287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189300.180286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189300.436243:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189300.693314:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189300.732655:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189300.948282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189301.204334:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189301.460479:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189301.716299:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189301.756485:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189301.972289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189302.141642:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189302.141692:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189302.141722:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189302.141757:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654950400 00000100:00100000:1.0:1776189302.141805:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1422 00000100:00100000:1.0:1776189302.141812:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d6b85e7e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654950400:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776189302.141873:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d6b85e7e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654950400:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 61us (150us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189302.141881:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1422 00000100:00100000:2.0:1776189302.142055:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654950464 00000100:00100000:2.0:1776189302.142088:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654950528 00000100:00100000:2.0:1776189302.142110:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9888 00000100:00100000:2.0:1776189302.142114:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a37ce050 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654950464:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776189302.142146:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a37ce050 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654950464:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 30us (512us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189302.142152:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9888 00000100:00100000:2.0:1776189302.142164:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57319 00000100:00100000:2.0:1776189302.142168:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c6d0dff pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654950528:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:2.0:1776189302.142224:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189302.142250:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c6d0dff pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654950528:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 80us (558us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189302.142256:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57319 00010000:02020000:0.0:1776189302.228377:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189302.463922:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776189302.463957:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a12b3db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664322688:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189302.464024:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189302.464211:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664322688 00000100:00100000:1.0:1776189302.464256:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6728 00000100:00100000:1.0:1776189302.464261:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b1b59934 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664322688:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189302.464288:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189302.464346:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b1b59934 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664322688:12345-0@lo:13:osp-pre-0-0.0 Request processed in 85us (323us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189302.464352:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6728 00000100:00100000:0.0:1776189302.464522:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a12b3db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664322688:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189302.468121:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d38ebc08 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664322752:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189302.468235:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189302.468348:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664322752 00000100:00100000:3.0:1776189302.468399:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6729 00000100:00100000:3.0:1776189302.468404:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000bfa9efc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664322752:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776189302.468435:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189302.468482:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000bfa9efc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664322752:12345-0@lo:13:osp-pre-1-0.0 Request processed in 77us (247us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189302.468488:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6729 00000100:00100000:2.0:1776189302.468818:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d38ebc08 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664322752:0@lo:13:osp-pre-1-0.0 00010000:02020000:0.0:1776189302.484390:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189302.740289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189302.780665:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189302.997356:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189303.252390:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189303.484486:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776189303.484507:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e3bf25aa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664322816:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189303.484574:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189303.484612:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664322816 00000100:00100000:1.0:1776189303.484647:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6730 00000100:00100000:1.0:1776189303.484652:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008530b2a3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664322816:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189303.484673:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189303.484713:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008530b2a3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664322816:12345-0@lo:13:osp-pre-1-1.0 Request processed in 60us (140us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189303.484719:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6730 00000100:00100000:1.0:1776189303.484764:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e3bf25aa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664322816:0@lo:13:osp-pre-1-1.0 00000100:00080000:3.0:1776189303.484863:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189303.484868:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189303.484871:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189303.484875:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9445 00000100:00080000:3.0:1776189303.484877:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776189303.484889:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189303.484906:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d0281336 x1862459664323008/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189303.484940:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189303.484943:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189303.484944:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189303.484947:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9445 00000100:00080000:3.0:1776189303.484949:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776189303.484952:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189303.484960:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a829435f x1862459664323136/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189303.484967:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189303.484969:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189303.484970:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189303.484973:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9445 00000100:00080000:3.0:1776189303.484975:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776189303.484977:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189303.484985:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005f6664e0 x1862459664323200/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189303.485157:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e3bf25aa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664322880:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189303.485217:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189303.485231:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e532ce7c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664323328:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189303.485254:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189303.485273:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1776189303.485278:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d8f74173 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664322944:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189303.485304:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189303.485319:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003cd25a35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664323392:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189303.485339:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189303.485363:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664323328 00000100:00100000:1.0:1776189303.485390:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8507 00000100:00100000:1.0:1776189303.485394:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003179aec9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664323328:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189303.485428:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003179aec9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664323328:12345-0@lo:400:kworker/u8:1.0 Request processed in 33us (174us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189303.485435:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8507 00000100:00100000:1.0:1776189303.485454:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664322944 00000100:00100000:1.0:1776189303.485481:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664323392 00000100:00100000:1.0:1776189303.485495:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9889 00000100:00100000:1.0:1776189303.485498:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006c19e1dc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664322944:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189303.485527:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006c19e1dc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664322944:12345-0@lo:400:kworker/u8:1.0 Request processed in 28us (223us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189303.485533:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9889 00000100:00100000:1.0:1776189303.485544:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9890 00000100:00100000:1.0:1776189303.485547:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004fc70dbb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664323392:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189303.485569:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004fc70dbb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664323392:12345-0@lo:400:kworker/u8:1.0 Request processed in 20us (229us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189303.485574:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9890 00000100:00100000:1.0:1776189303.485608:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d8f74173 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664322944:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189303.485618:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003cd25a35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664323392:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189303.485625:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e532ce7c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664323328:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189303.485649:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b54d6c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664323072:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189303.485670:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189303.485684:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038a7e045 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664323456:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189303.485702:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189303.485715:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776189303.485720:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000032d9de2c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664323264:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189303.485737:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189303.485756:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664323456 00000100:00100000:1.0:1776189303.485776:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664323264 00000100:00100000:1.0:1776189303.485789:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8508 00000100:00100000:1.0:1776189303.485792:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003a195883 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664323456:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189303.485820:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003a195883 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664323456:12345-0@lo:400:kworker/u8:1.0 Request processed in 27us (118us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189303.485825:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8508 00000100:00100000:1.0:1776189303.485836:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8509 00000100:00100000:1.0:1776189303.485839:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003abc2764 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664323264:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189303.485861:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003abc2764 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664323264:12345-0@lo:400:kworker/u8:1.0 Request processed in 21us (124us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189303.485866:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8509 00000100:00100000:1.0:1776189303.485892:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664323072 00000100:00100000:1.0:1776189303.485911:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9891 00000100:00100000:1.0:1776189303.485914:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001e0f6d29 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664323072:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189303.486033:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664322880 00000100:00100000:3.0:1776189303.486062:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1423 00000100:00100000:3.0:1776189303.486065:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004c446be7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664322880:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189303.486100:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004c446be7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664322880:12345-0@lo:400:kworker/u8:1.0 Request processed in 34us (883us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189303.486109:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1423 00000100:00100000:3.0:1776189303.486150:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e3bf25aa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664322880:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189303.486306:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001e0f6d29 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664323072:12345-0@lo:400:kworker/u8:1.0 Request processed in 389us (634us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189303.486313:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9891 00000100:00100000:1.0:1776189303.486338:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000032d9de2c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664323264:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189303.486347:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b54d6c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664323072:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189303.486355:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038a7e045 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664323456:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189303.486508:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776189303.486516:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e310c888 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664323520:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189303.486540:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189303.487043:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d0281336 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664323008:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189303.487075:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189303.487100:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a829435f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664323136:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189303.487120:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189303.487143:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005f6664e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664323200:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189303.487167:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189303.488037:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664323200 00000100:00100000:2.0:1776189303.488076:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9106 00000100:00100000:2.0:1776189303.488081:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ed846a61 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664323200:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189303.488096:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189303.488138:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ed846a61 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664323200:12345-0@lo:38:kworker/u8:1.0 Request processed in 57us (971us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189303.488145:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9106 00000100:00100000:2.0:1776189303.488167:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664323008 00000100:00100000:2.0:1776189303.488210:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664323136 00000100:00100000:2.0:1776189303.488228:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57320 00000100:00100000:2.0:1776189303.488232:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009be3a12f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664323008:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189303.488239:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189303.488263:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009be3a12f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664323008:12345-0@lo:38:kworker/u8:1.0 Request processed in 30us (1189us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189303.488268:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57320 00000100:00100000:2.0:1776189303.488280:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57321 00000100:00100000:2.0:1776189303.488283:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dabf4d62 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664323136:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189303.488289:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189303.488307:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dabf4d62 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664323136:12345-0@lo:38:kworker/u8:1.0 Request processed in 24us (1187us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189303.488312:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57321 00000100:00080000:0.0:1776189303.488403:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189303.488408:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189303.488412:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d0281336 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664323008:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776189303.488422:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189303.488425:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189303.488427:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a829435f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664323136:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776189303.488434:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189303.488436:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189303.488438:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f6664e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664323200:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189303.492719:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664323520 00000100:00100000:3.0:1776189303.492787:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6731 00000100:00100000:3.0:1776189303.492793:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008e3febe1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664323520:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776189303.492820:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189303.492919:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008e3febe1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664323520:12345-0@lo:13:osp-pre-0-1.0 Request processed in 125us (6377us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189303.492933:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6731 00000100:00100000:0.0:1776189303.493156:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e310c888 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664323520:0@lo:13:osp-pre-0-1.0 00010000:02020000:0.0:1776189303.508298:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189303.769833:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189303.806694:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189304.020522:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189304.276279:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189304.532286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189304.788294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189304.831951:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189305.044313:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189305.302392:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189305.556272:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189305.812331:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189305.855726:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189306.068287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189306.324328:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189306.580336:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189306.836339:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189306.880795:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189307.092239:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189307.258294:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189307.258352:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189307.258482:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654950656 00000100:00100000:1.0:1776189307.258536:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9892 00000100:00100000:1.0:1776189307.258543:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f6664e0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654950656:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776189307.258622:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f6664e0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654950656:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 79us (270us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189307.258629:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9892 00000100:00100000:3.0:1776189307.258915:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654950592 00000100:00100000:2.0:1776189307.258981:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189307.259003:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1424 00000100:00100000:2.0:1776189307.259009:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654950720 00000100:00100000:2.0:1776189307.259030:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57322 00000100:00100000:2.0:1776189307.259034:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005bd91e44 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654950720:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:2.0:1776189307.259048:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189307.259077:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005bd91e44 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654950720:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 41us (96us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189307.259081:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fe4b4330 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654950592:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776189307.259086:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57322 00000100:00100000:3.0:1776189307.259120:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fe4b4330 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654950592:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 109us (844us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189307.259127:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1424 00010000:02020000:2.0:1776189307.348284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189307.580575:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fd39f2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664323584:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189307.580650:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189307.583644:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776189307.583664:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d6111a35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664323648:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189307.583737:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189307.583783:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664323584 00000100:00100000:1.0:1776189307.583835:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664323648 00000100:00100000:1.0:1776189307.583852:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6732 00000100:00100000:1.0:1776189307.583858:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c27afb9e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664323584:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189307.583886:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189307.583926:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c27afb9e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664323584:12345-0@lo:13:osp-pre-1-0.0 Request processed in 67us (3277us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189307.583934:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6732 00000100:00100000:1.0:1776189307.583951:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6733 00000100:00100000:1.0:1776189307.583955:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d7c70f6f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664323648:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189307.583968:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189307.584007:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d7c70f6f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664323648:12345-0@lo:13:osp-pre-0-0.0 Request processed in 51us (271us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189307.584012:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6733 00000100:00100000:1.0:1776189307.584061:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d6111a35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664323648:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189307.584077:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fd39f2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664323584:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189307.604294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189307.861538:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189307.901588:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189308.119095:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189308.372287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189308.604459:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c3b654b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664323712:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189308.604527:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1776189308.604808:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189308.604813:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189308.604816:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189308.604821:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9450 00000100:00080000:3.0:1776189308.604824:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00080000:3.0:1776189308.604829:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189308.604846:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000305a0bea x1862459664323904/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189308.604865:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189308.604868:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189308.604869:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189308.604872:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9450 00000100:00080000:3.0:1776189308.604874:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00080000:3.0:1776189308.604888:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189308.604897:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009f7be854 x1862459664324032/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189308.604904:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189308.604907:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189308.604908:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189308.604910:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9450 00000100:00080000:3.0:1776189308.604912:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 6s 00000100:00080000:3.0:1776189308.604914:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189308.604924:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000efff0bc3 x1862459664324096/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189308.604963:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fbad805c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664324160:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189308.604992:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189308.605011:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1776189308.605016:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f40573b9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664323776:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189308.605055:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189308.605070:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e0c8fd67 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664324224:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189308.605089:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189308.605117:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664324160 00000100:00100000:3.0:1776189308.605152:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664324224 00000100:00100000:3.0:1776189308.605186:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8510 00000100:00100000:3.0:1776189308.605192:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c68cc49e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664324160:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189308.605237:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c68cc49e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664324160:12345-0@lo:400:kworker/u8:1.0 Request processed in 45us (245us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189308.605244:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8510 00000100:00100000:3.0:1776189308.605259:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8511 00000100:00100000:3.0:1776189308.605262:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000002d6a3fa pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664324224:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189308.605283:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000002d6a3fa pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664324224:12345-0@lo:400:kworker/u8:1.0 Request processed in 21us (194us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189308.605289:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8511 00000100:00100000:3.0:1776189308.605305:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664323776 00000100:00100000:3.0:1776189308.605325:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1425 00000100:00100000:3.0:1776189308.605328:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000459e616f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664323776:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189308.605354:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000459e616f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664323776:12345-0@lo:400:kworker/u8:1.0 Request processed in 25us (299us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189308.605361:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1425 00000100:00100000:3.0:1776189308.605403:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f40573b9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664323776:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189308.605415:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e0c8fd67 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664324224:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189308.605423:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fbad805c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664324160:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189308.605478:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000838deece pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664323840:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189308.605502:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189308.605518:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba5f0e8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664324288:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189308.605536:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189308.605553:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1776189308.605558:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c04acb9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664323968:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189308.605591:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189308.605605:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f8dcf939 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664324352:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189308.605624:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189308.605645:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664324352 00000100:00100000:3.0:1776189308.605669:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8512 00000100:00100000:3.0:1776189308.605673:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c3d6c201 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664324352:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189308.605703:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c3d6c201 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664324352:12345-0@lo:400:kworker/u8:1.0 Request processed in 29us (79us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189308.605707:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8512 00000100:00100000:3.0:1776189308.605728:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664323840 00000100:00100000:3.0:1776189308.605748:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664324288 00000100:00100000:3.0:1776189308.605763:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664323968 00000100:00100000:3.0:1776189308.605788:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9893 00000100:00100000:3.0:1776189308.605792:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fbad805c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664323840:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189308.605820:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fbad805c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664323840:12345-0@lo:400:kworker/u8:1.0 Request processed in 27us (318us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189308.605826:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9893 00000100:00100000:3.0:1776189308.605836:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9894 00000100:00100000:3.0:1776189308.605839:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e0c8fd67 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664324288:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189308.605860:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e0c8fd67 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664324288:12345-0@lo:400:kworker/u8:1.0 Request processed in 20us (324us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189308.605865:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9894 00000100:00100000:3.0:1776189308.605874:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9895 00000100:00100000:3.0:1776189308.605877:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f40573b9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664323968:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189308.605898:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f40573b9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664323968:12345-0@lo:400:kworker/u8:1.0 Request processed in 20us (308us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189308.605903:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9895 00000100:00100000:3.0:1776189308.605936:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c04acb9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664323968:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189308.605944:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f8dcf939 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664324352:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189308.605951:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000838deece pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664323840:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189308.605958:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba5f0e8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664324288:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189308.606260:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000305a0bea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664323904:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189308.606317:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189308.606344:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009f7be854 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664324032:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189308.606367:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189308.606381:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000efff0bc3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664324096:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189308.606407:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189308.606435:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664324096 00000100:00100000:0.0:1776189308.606455:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9107 00000100:00100000:0.0:1776189308.606458:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d72c99d5 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664324096:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189308.606470:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189308.606500:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d72c99d5 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664324096:12345-0@lo:38:kworker/u8:1.0 Request processed in 41us (93us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189308.606506:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9107 00000100:00100000:0.0:1776189308.606518:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776189308.606525:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b864aa6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664324416:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189308.606547:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189308.606619:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664323904 00000100:00100000:3.0:1776189308.606636:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664324032 00000100:00100000:3.0:1776189308.606757:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57323 00000100:00100000:3.0:1776189308.606761:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ced31f93 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664323904:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189308.606782:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189308.606808:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ced31f93 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664323904:12345-0@lo:38:kworker/u8:1.0 Request processed in 45us (491us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189308.606813:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57323 00000100:00100000:3.0:1776189308.606824:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57324 00000100:00100000:3.0:1776189308.606827:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000040e81793 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664324032:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189308.606833:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189308.606853:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000040e81793 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664324032:12345-0@lo:38:kworker/u8:1.0 Request processed in 25us (486us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189308.606858:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57324 00000100:00080000:3.0:1776189308.607221:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189308.607225:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189308.607229:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000305a0bea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664323904:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776189308.607241:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189308.607244:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189308.607246:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009f7be854 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664324032:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776189308.607254:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189308.607256:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189308.607258:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000efff0bc3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664324096:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189308.613264:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664323712 00000100:00100000:3.0:1776189308.613325:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664324416 00000100:00100000:3.0:1776189308.613343:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6734 00000100:00100000:3.0:1776189308.613348:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cc4b762b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664323712:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776189308.613372:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189308.613436:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cc4b762b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664323712:12345-0@lo:13:osp-pre-0-1.0 Request processed in 87us (8910us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189308.613442:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6734 00000100:00100000:3.0:1776189308.613458:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6735 00000100:00100000:3.0:1776189308.613462:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0fb2966 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664324416:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189308.613472:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189308.613498:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0fb2966 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664324416:12345-0@lo:13:osp-pre-1-1.0 Request processed in 35us (6951us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189308.613503:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6735 00000100:00100000:3.0:1776189308.613545:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c3b654b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664323712:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189308.613643:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b864aa6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664324416:0@lo:13:osp-pre-1-1.0 00010000:02020000:2.0:1776189308.628654:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189308.885362:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189308.924477:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189309.141041:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189309.396322:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189309.652300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189309.908308:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189309.948866:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189310.164253:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189310.420256:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189310.676281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189310.932272:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189310.974564:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189311.188299:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189311.449975:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189311.700279:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189311.956282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189311.996761:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189312.212318:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189312.380374:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189312.383163:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654950784 00000100:00100000:3.0:1776189312.383256:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1426 00000100:00100000:3.0:1776189312.383262:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000de62cc6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654950784:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189312.383926:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000de62cc6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654950784:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 661us (3558us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189312.383935:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1426 00000100:00100000:0.0:1776189312.384475:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189312.384776:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189312.384812:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654950848 00000100:00100000:0.0:1776189312.384848:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654950912 00000100:00100000:0.0:1776189312.384867:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9896 00000100:00100000:0.0:1776189312.384873:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009b827fed pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654950848:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189312.384896:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57325 00000100:00100000:3.0:1776189312.384902:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000033264082 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654950912:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:3.0:1776189312.384920:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189312.384932:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009b827fed pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654950848:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 57us (458us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189312.384943:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9896 00000100:00100000:3.0:1776189312.384964:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000033264082 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654950912:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 59us (387us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189312.384972:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57325 00010000:02020000:2.0:1776189312.468289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189312.700466:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776189312.700489:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d6111a35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664324480:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189312.700569:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189312.700726:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664324480 00000100:00100000:1.0:1776189312.700772:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6736 00000100:00100000:1.0:1776189312.700778:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000efff0bc3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664324480:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189312.700806:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189312.700855:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000efff0bc3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664324480:12345-0@lo:13:osp-pre-0-0.0 Request processed in 77us (287us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189312.700861:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6736 00000100:00100000:1.0:1776189312.701239:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776189312.701251:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009015c790 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664324544:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189312.701280:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189312.701391:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d6111a35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664324480:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189312.701424:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664324544 00000100:00100000:2.0:1776189312.701460:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6737 00000100:00100000:2.0:1776189312.701465:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d7d9005e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664324544:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189312.701482:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189312.701515:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d7d9005e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664324544:12345-0@lo:13:osp-pre-1-0.0 Request processed in 49us (235us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189312.701521:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6737 00000100:00100000:1.0:1776189312.701659:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009015c790 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664324544:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189312.728572:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189312.980547:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189313.022342:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189313.236393:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189313.493201:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189313.724506:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b864aa6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664324608:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189313.724574:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189313.724685:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664324608 00000100:00100000:1.0:1776189313.724690:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017f4990e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664324672:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189313.724720:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189313.724745:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664324672 00000100:00100000:1.0:1776189313.724760:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6738 00000100:00100000:1.0:1776189313.724766:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f9b443b4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664324608:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189313.724866:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189313.724904:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f9b443b4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664324608:12345-0@lo:13:osp-pre-1-1.0 Request processed in 137us (330us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189313.729576:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6739 00000100:00100000:3.0:1776189313.729590:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e60f52de pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664324672:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776189313.729626:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00080000:2.0:1776189313.729722:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189313.729727:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189313.729730:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189313.729736:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9455 00000100:00080000:2.0:1776189313.729739:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00100000:3.0:1776189313.729742:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e60f52de pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664324672:12345-0@lo:13:osp-pre-0-1.0 Request processed in 152us (5019us total) trans 0 rc 0/0 00000100:00080000:2.0:1776189313.729746:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189313.729754:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6739 00000100:00100000:2.0:1776189313.729767:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003b5d789a x1862459664324864/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189313.729800:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189313.729803:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189313.729805:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1776189313.729806:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003b5d789a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664324864:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776189313.729811:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9455 00000100:00080000:2.0:1776189313.729814:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:2.0:1776189313.729817:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189313.729830:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cccdb2ad x1862459664324992/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189313.729838:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189313.729841:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189313.729843:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189313.729846:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9455 00000100:00100000:3.0:1776189313.729861:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189313.729888:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cccdb2ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664324992:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189313.729915:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189313.729949:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664324864 00000100:00100000:3.0:1776189313.729976:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664324992 00000100:00100000:3.0:1776189313.729993:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57326 00000100:00100000:3.0:1776189313.729996:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009f7be854 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664324864:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189313.730026:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189313.730061:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009f7be854 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664324864:12345-0@lo:38:kworker/u8:0.0 Request processed in 62us (213us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189313.730066:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57326 00000100:00100000:3.0:1776189313.730076:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57327 00000100:00100000:3.0:1776189313.730079:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000305a0bea pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664324992:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189313.730084:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189313.730103:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000305a0bea pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664324992:12345-0@lo:38:kworker/u8:0.0 Request processed in 24us (189us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189313.730109:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57327 00000100:00080000:3.0:1776189313.730144:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189313.730148:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189313.730152:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cccdb2ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664324992:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776189313.730160:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189313.730162:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189313.730165:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003b5d789a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664324864:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776189313.730830:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 11s 00000100:00080000:2.0:1776189313.730836:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189313.730871:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b45ebec6 x1862459664325056/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189313.730911:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b45ebec6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664325056:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189313.730938:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002e4c5085 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664324800:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189313.730955:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189313.731502:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6738 00000100:00100000:1.0:1776189313.731541:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002460f3d1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664324928:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189313.731628:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664325056 00000100:00100000:0.0:1776189313.731655:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b0629a6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664324736:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189313.731655:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9108 00000100:00100000:3.0:1776189313.731674:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba5f0e8e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664325056:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189313.731687:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189313.731705:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189313.731734:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b9092100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664325184:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189313.731743:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba5f0e8e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664325056:12345-0@lo:38:kworker/u8:0.0 Request processed in 67us (788us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189313.731749:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9108 00000100:00100000:3.0:1776189313.731771:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664324736 00000100:00100000:0.0:1776189313.731783:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189313.731799:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1427 00000100:00100000:3.0:1776189313.731804:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000082b19b4e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664324736:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189313.731811:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776189313.731819:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b385e94b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664325120:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189313.731840:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000082b19b4e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664324736:12345-0@lo:400:kworker/u8:0.0 Request processed in 35us (136us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189313.731847:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1427 00000100:00080000:3.0:1776189313.731880:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189313.731883:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189313.731886:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b45ebec6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664325056:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189313.731948:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017f4990e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664324672:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189313.731980:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664325184 00000100:00100000:3.0:1776189313.732003:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8513 00000100:00100000:3.0:1776189313.732017:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001aa39bf8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664325184:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189313.732050:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001aa39bf8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664325184:12345-0@lo:400:kworker/u8:0.0 Request processed in 42us (268us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189313.732056:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8513 00000100:00100000:0.0:1776189313.732062:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189313.732096:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b0629a6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664324736:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189313.732106:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b9092100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664325184:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189313.732153:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189313.732187:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000048520f37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664325248:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189313.732212:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189313.732240:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664324800 00000100:00100000:2.0:1776189313.732263:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664325248 00000100:00100000:2.0:1776189313.732278:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9897 00000100:00100000:2.0:1776189313.732282:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000db3523c9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664324800:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189313.732315:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000db3523c9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664324800:12345-0@lo:400:kworker/u8:0.0 Request processed in 32us (163us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189313.732321:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9897 00000100:00100000:2.0:1776189313.732332:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9898 00000100:00100000:2.0:1776189313.732336:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000060a8344d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664325248:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189313.732359:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000060a8344d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664325248:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (147us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189313.732364:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9898 00000100:00100000:2.0:1776189313.732395:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002e4c5085 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664324800:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189313.732402:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000048520f37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664325248:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189313.732578:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189313.732613:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b864aa6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664324608:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189313.732703:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664324928 00000100:00100000:2.0:1776189313.732725:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9899 00000100:00100000:2.0:1776189313.732728:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f4d61fa4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664324928:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189313.732760:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f4d61fa4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664324928:12345-0@lo:400:kworker/u8:0.0 Request processed in 31us (183us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189313.732770:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9899 00000100:00100000:2.0:1776189313.732795:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002460f3d1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664324928:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189313.735575:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664325120 00000100:00100000:3.0:1776189313.735645:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8514 00000100:00100000:3.0:1776189313.735651:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000aecac7d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664325120:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189313.735764:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000aecac7d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664325120:12345-0@lo:400:kworker/u8:0.0 Request processed in 111us (3913us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189313.735774:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8514 00000100:00100000:0.0:1776189313.735806:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b385e94b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664325120:0@lo:400:kworker/u8:0.0 00010000:02020000:2.0:1776189313.748614:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189314.004443:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189314.044682:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189314.260311:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189314.517091:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189314.772659:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189315.029651:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189315.069079:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189315.284294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189315.540578:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189315.796863:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189316.053227:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189316.092843:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189316.308349:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189316.564531:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189316.820353:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189317.076312:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189317.117340:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189317.334796:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189317.499401:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189317.499466:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654951040 00000100:00100000:2.0:1776189317.499470:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189317.499522:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9900 00000100:00100000:0.0:1776189317.499529:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002460f3d1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654951040:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776189317.499552:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189317.499592:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654951104 00000100:00100000:0.0:1776189317.499607:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002460f3d1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654951040:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 76us (214us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189317.499617:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9900 00000100:00100000:2.0:1776189317.499623:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57328 00000100:00100000:2.0:1776189317.499629:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000048520f37 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654951104:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:2.0:1776189317.499645:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189317.499676:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000048520f37 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654951104:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 46us (126us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189317.499683:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57328 00000100:00100000:1.0:1776189317.499897:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654950976 00000100:00100000:1.0:1776189317.499930:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1428 00000100:00100000:1.0:1776189317.499934:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000055012753 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654950976:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776189317.499966:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000055012753 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654950976:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 32us (510us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189317.499972:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1428 00010000:02020000:2.0:1776189317.588286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189317.820440:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776189317.820464:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002e4c5085 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664325312:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189317.820466:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776189317.820479:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000838deece pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664325376:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189317.823678:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189317.823835:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189317.824114:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664325376 00000100:00100000:2.0:1776189317.824470:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664325312 00000100:00100000:2.0:1776189317.824507:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6740 00000100:00100000:2.0:1776189317.824513:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f8dcf939 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664325376:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189317.824546:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189317.824611:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f8dcf939 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664325376:12345-0@lo:13:osp-pre-0-0.0 Request processed in 97us (943us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189317.824618:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6740 00000100:00100000:2.0:1776189317.824710:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6741 00000100:00100000:2.0:1776189317.824715:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000485a8bc1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664325312:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189317.824731:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189317.824765:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000485a8bc1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664325312:12345-0@lo:13:osp-pre-1-0.0 Request processed in 50us (932us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189317.824772:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6741 00000100:00100000:2.0:1776189317.824832:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002e4c5085 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664325312:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189317.825164:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000838deece pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664325376:0@lo:13:osp-pre-0-0.0 00010000:02020000:2.0:1776189317.844463:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189318.100374:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189318.140436:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189318.356277:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189318.612295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:0.0:1776189318.844392:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189318.844398:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189318.844401:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189318.844409:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9460 00000100:00080000:0.0:1776189318.844412:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:0.0:1776189318.844417:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189318.844434:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003f21295b x1862459664325568/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189318.844436:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00080000:0.0:1776189318.844458:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189318.844462:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189318.844464:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1776189318.844464:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000053ddeb10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664325440:0@lo:400:kworker/u8:1.0 00000100:00080000:0.0:1776189318.844471:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9460 00000100:00080000:0.0:1776189318.844473:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00080000:0.0:1776189318.844476:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189318.844485:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000075ad1ba3 x1862459664325696/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189318.844492:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189318.844495:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189318.844496:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189318.844499:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9460 00000100:00080000:0.0:1776189318.844501:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00080000:0.0:1776189318.844503:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189318.844512:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cda39ea2 x1862459664325760/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189318.844535:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189318.844575:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6402bd2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664325824:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189318.844592:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003f21295b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664325568:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189318.844607:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189318.844626:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007c812403 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664326080:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189318.844632:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189318.844653:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000075ad1ba3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664325696:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189318.844660:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189318.844679:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776189318.844685:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e8ddaf27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664325888:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189318.844692:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189318.844709:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cda39ea2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664325760:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189318.844717:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189318.844738:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189318.844756:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664325568 00000100:00100000:0.0:1776189318.844772:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664325824 00000100:00100000:3.0:1776189318.844787:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664325696 00000100:00100000:0.0:1776189318.844806:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664326080 00000100:00100000:3.0:1776189318.844813:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57329 00000100:00100000:3.0:1776189318.844820:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000048a53cb2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664325568:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189318.844829:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664325888 00010000:02000000:3.0:1776189318.844836:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189318.844848:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8515 00000100:00100000:0.0:1776189318.844852:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c04acb9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664325824:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189318.844885:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000048a53cb2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664325568:12345-0@lo:38:kworker/u8:1.0 Request processed in 64us (253us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189318.844894:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57329 00000100:00100000:0.0:1776189318.844902:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c04acb9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664325824:12345-0@lo:400:kworker/u8:1.0 Request processed in 47us (296us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189318.844911:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8515 00000100:00100000:3.0:1776189318.844915:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57330 00000100:00100000:3.0:1776189318.844921:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005c6b517b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664325696:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189318.844927:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8516 00010000:02000000:3.0:1776189318.844930:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189318.844934:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ed29900a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664326080:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189318.844967:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005c6b517b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664325696:12345-0@lo:38:kworker/u8:1.0 Request processed in 44us (275us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189318.844972:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ed29900a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664326080:12345-0@lo:400:kworker/u8:1.0 Request processed in 37us (314us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189318.844978:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57330 00000100:00100000:0.0:1776189318.844981:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8516 00000100:00100000:0.0:1776189318.844996:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8517 00000100:00100000:0.0:1776189318.845000:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008206d902 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664325888:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189318.845025:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008206d902 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664325888:12345-0@lo:400:kworker/u8:1.0 Request processed in 25us (310us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189318.845026:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6402bd2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664325824:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189318.845035:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8517 00000100:00100000:3.0:1776189318.845046:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007c812403 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664326080:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189318.845057:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664325760 00000100:00100000:3.0:1776189318.845074:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e8ddaf27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664325888:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189318.845079:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9109 00000100:00100000:0.0:1776189318.845084:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006047c8f7 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664325760:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189318.845092:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189318.845115:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006047c8f7 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664325760:12345-0@lo:38:kworker/u8:1.0 Request processed in 31us (379us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189318.845125:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9109 00000100:00080000:0.0:1776189318.845148:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189318.845151:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189318.845154:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003f21295b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664325568:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776189318.845160:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189318.845162:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189318.845164:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000075ad1ba3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664325696:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776189318.845170:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189318.845202:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189318.845204:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cda39ea2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664325760:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189318.845434:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664325440 00000100:00100000:2.0:1776189318.845467:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1429 00000100:00100000:2.0:1776189318.845470:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bffe4051 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664325440:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189318.845508:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bffe4051 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664325440:12345-0@lo:400:kworker/u8:1.0 Request processed in 37us (976us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189318.845514:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1429 00000100:00100000:2.0:1776189318.845544:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005f3b872c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664325504:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189318.845591:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000053ddeb10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664325440:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189318.845594:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189318.845613:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8bc9a84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664325952:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189318.845628:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664325504 00000100:00100000:2.0:1776189318.845636:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189318.845658:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664325952 00000100:00100000:2.0:1776189318.845666:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1776189318.845675:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000006c1656a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664325632:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189318.845679:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9901 00000100:00100000:3.0:1776189318.845686:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002e4c5085 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664325504:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189318.845702:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189318.845725:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b9f0254e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664326016:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189318.845731:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002e4c5085 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664325504:12345-0@lo:400:kworker/u8:1.0 Request processed in 44us (140us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189318.845739:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9901 00000100:00100000:2.0:1776189318.845752:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189318.845753:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664325632 00000100:00100000:2.0:1776189318.845778:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f3b872c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664325504:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189318.845781:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9902 00000100:00100000:3.0:1776189318.845788:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e6665a35 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664325952:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189318.845809:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9903 00000100:00100000:2.0:1776189318.845815:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008cc6ee78 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664325632:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189318.845824:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e6665a35 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664325952:12345-0@lo:400:kworker/u8:1.0 Request processed in 35us (188us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189318.845832:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9902 00000100:00100000:2.0:1776189318.845854:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008cc6ee78 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664325632:12345-0@lo:400:kworker/u8:1.0 Request processed in 38us (152us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189318.845861:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9903 00000100:00100000:3.0:1776189318.845874:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664326016 00000100:00100000:2.0:1776189318.845893:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000006c1656a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664325632:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189318.845904:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8518 00000100:00100000:2.0:1776189318.845908:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8bc9a84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664325952:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189318.845912:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000739e313a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664326016:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189318.845954:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000739e313a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664326016:12345-0@lo:400:kworker/u8:1.0 Request processed in 45us (203us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189318.845978:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b9f0254e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664326016:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189318.846009:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8518 00000100:00100000:1.0:1776189318.847084:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776189318.847098:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009a571ce7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664326144:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189318.847138:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189318.847186:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664326144 00000100:00100000:1.0:1776189318.847218:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6742 00000100:00100000:1.0:1776189318.847221:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003d7747fc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664326144:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189318.847238:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189318.847273:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003d7747fc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664326144:12345-0@lo:13:osp-pre-0-1.0 Request processed in 51us (136us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189318.847280:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6742 00000100:00100000:1.0:1776189318.847317:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009a571ce7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664326144:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189318.847595:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006302d616 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664326208:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189318.847623:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189318.847694:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664326208 00000100:00100000:0.0:1776189318.847721:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6743 00000100:00100000:0.0:1776189318.847724:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009a571ce7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664326208:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189318.847734:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189318.847771:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009a571ce7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664326208:12345-0@lo:13:osp-pre-1-1.0 Request processed in 45us (147us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189318.847776:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6743 00000100:00100000:3.0:1776189318.847838:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006302d616 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664326208:0@lo:13:osp-pre-1-1.0 00010000:02020000:2.0:1776189318.868747:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189319.124273:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189319.167053:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189319.380300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189319.636470:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189319.892280:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189320.148290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189320.188436:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189320.404290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189320.660646:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189320.916288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189321.172285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189321.212561:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189321.428322:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189321.684280:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189321.940288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189322.196309:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189322.236760:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189322.454271:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189322.620072:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189322.620281:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189322.620357:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189322.620418:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654951168 00000100:00100000:0.0:1776189322.620479:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1430 00000100:00100000:0.0:1776189322.620485:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002aae1640 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654951168:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189322.620552:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002aae1640 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654951168:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 67us (273us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189322.620559:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1430 00000100:00100000:3.0:1776189322.624654:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654951232 00000100:00100000:0.0:1776189322.624684:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654951296 00000100:00100000:3.0:1776189322.624727:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9904 00000100:00100000:3.0:1776189322.624735:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c76be6ad pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654951232:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189322.624806:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57331 00000100:00100000:3.0:1776189322.624816:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c76be6ad pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654951232:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 80us (4751us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189322.624855:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000822c622a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654951296:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776189322.624871:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189322.624902:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000822c622a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654951296:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 50us (4546us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189322.624914:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57331 00000100:00100000:3.0:1776189322.630021:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9904 00010000:02020000:2.0:1776189322.708567:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189322.942163:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776189322.942202:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b385e94b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664326272:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189322.942263:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189322.942415:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006302d616 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664326336:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189322.942440:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189322.942597:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664326272 00000100:00100000:3.0:1776189322.942640:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664326336 00000100:00100000:3.0:1776189322.942655:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6744 00000100:00100000:3.0:1776189322.942660:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000757dd227 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664326272:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776189322.942686:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189322.942719:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000757dd227 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664326272:12345-0@lo:13:osp-pre-1-0.0 Request processed in 58us (457us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189322.942735:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6744 00000100:00100000:3.0:1776189322.942750:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6745 00000100:00100000:3.0:1776189322.942753:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000017f4990e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664326336:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189322.942766:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189322.942786:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000017f4990e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664326336:12345-0@lo:13:osp-pre-0-0.0 Request processed in 32us (347us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189322.942791:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6745 00000100:00100000:1.0:1776189322.942931:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006302d616 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664326336:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189322.942957:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b385e94b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664326272:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189322.964738:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189323.220285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189323.261738:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189323.476280:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189323.732388:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189323.968225:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000068da6036 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664326400:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189323.968295:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189323.968670:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664326400 00000100:00100000:3.0:1776189323.968734:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6746 00000100:00100000:3.0:1776189323.968740:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cda39ea2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664326400:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189323.968769:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189323.968825:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cda39ea2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664326400:12345-0@lo:13:osp-pre-1-1.0 Request processed in 84us (531us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189323.968833:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6746 00000100:00080000:2.0:1776189323.968878:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1776189323.968881:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b9092100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664326464:0@lo:400:kworker/u8:0.0 00000100:00080000:2.0:1776189323.968885:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189323.968900:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189323.968906:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9465 00000100:00080000:2.0:1776189323.968910:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776189323.968915:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189323.968920:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189323.968937:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b45ebec6 x1862459664326592/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189323.968939:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776189323.968947:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b0629a6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664326528:0@lo:400:kworker/u8:0.0 00000100:00080000:2.0:1776189323.968967:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189323.968971:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189323.968972:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189323.968977:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9465 00000100:00100000:3.0:1776189323.968979:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1776189323.968981:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776189323.968985:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189323.968998:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cccdb2ad x1862459664326720/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189323.969007:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189323.969010:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189323.969012:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1776189323.969012:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664326528 00000100:00080000:2.0:1776189323.969018:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9465 00000100:00080000:2.0:1776189323.969020:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776189323.969023:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189323.969034:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009015c790 x1862459664326784/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189323.969045:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9905 00000100:00100000:3.0:1776189323.969051:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000019f375fb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664326528:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189323.969101:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fd39f2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664326912:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189323.969112:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000019f375fb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664326528:12345-0@lo:400:kworker/u8:0.0 Request processed in 60us (136us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189323.969120:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9905 00000100:00100000:2.0:1776189323.969138:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189323.969160:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664326912 00000100:00100000:3.0:1776189323.969208:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8519 00000100:00100000:3.0:1776189323.969212:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b385e94b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664326912:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189323.969243:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b385e94b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664326912:12345-0@lo:400:kworker/u8:0.0 Request processed in 30us (107us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189323.969249:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8519 00000100:00100000:2.0:1776189323.969363:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b0629a6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664326528:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189323.969389:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fd39f2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664326912:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189323.969396:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776189323.969400:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e310c888 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664326976:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189323.969421:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189323.969743:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664326976 00000100:00100000:3.0:1776189323.969776:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9906 00000100:00100000:0.0:1776189323.969817:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b45ebec6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664326592:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189323.969845:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189323.969860:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cccdb2ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664326720:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189323.969880:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189323.969907:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009015c790 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664326784:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189323.969934:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189323.969953:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664326592 00000100:00100000:0.0:1776189323.969967:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664326784 00000100:00100000:0.0:1776189323.969993:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9110 00000100:00100000:0.0:1776189323.969998:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003ff7f95c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664326784:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189323.970025:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189323.970056:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003ff7f95c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664326784:12345-0@lo:38:kworker/u8:0.0 Request processed in 57us (123us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189323.970063:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9110 00000100:00080000:0.0:1776189323.970103:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189323.970106:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189323.970109:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009015c790 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664326784:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189323.970129:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664326464 00000100:00100000:0.0:1776189323.970151:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1431 00000100:00100000:0.0:1776189323.970154:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006f66ddc9 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664326464:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189323.970363:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006f66ddc9 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664326464:12345-0@lo:400:kworker/u8:0.0 Request processed in 207us (1447us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189323.970370:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1431 00000100:00100000:0.0:1776189323.970404:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b9092100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664326464:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189323.970423:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c3b654b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664326848:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189323.970458:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189323.970488:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000068da6036 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664326400:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189323.970501:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1776189323.970506:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003b5d789a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664326656:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189323.970526:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189323.970541:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c5b17f98 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664327040:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189323.970561:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189323.970584:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664326848 00000100:00100000:0.0:1776189323.970604:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664327040 00000100:00100000:0.0:1776189323.970620:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8520 00000100:00100000:0.0:1776189323.970624:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003a31bce8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664326848:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189323.970669:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003a31bce8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664326848:12345-0@lo:400:kworker/u8:0.0 Request processed in 44us (211us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189323.970675:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8520 00000100:00100000:0.0:1776189323.970686:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8521 00000100:00100000:0.0:1776189323.970689:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000349ae89b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664327040:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189323.970713:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000349ae89b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664327040:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (152us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189323.970718:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8521 00000100:00100000:1.0:1776189323.971198:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664326656 00000100:00100000:1.0:1776189323.971225:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9907 00000100:00100000:1.0:1776189323.971229:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d9a1a334 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664326656:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189323.971268:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d9a1a334 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664326656:12345-0@lo:400:kworker/u8:0.0 Request processed in 38us (742us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189323.971275:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9907 00000100:00100000:1.0:1776189323.971289:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57332 00000100:00100000:1.0:1776189323.971292:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000075ad1ba3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664326592:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189323.971303:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189323.971335:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000075ad1ba3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664326592:12345-0@lo:38:kworker/u8:0.0 Request processed in 42us (1490us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189323.971341:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57332 00000100:00100000:1.0:1776189323.971386:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776189323.971395:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d6111a35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664327104:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189323.971433:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189323.971467:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664327104 00000100:00100000:1.0:1776189323.971491:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6747 00000100:00100000:1.0:1776189323.971494:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d38ebc08 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664327104:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189323.971507:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189323.971538:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d38ebc08 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664327104:12345-0@lo:13:osp-pre-0-1.0 Request processed in 42us (105us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189323.971544:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6747 00000100:00100000:1.0:1776189323.971573:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d6111a35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664327104:0@lo:13:osp-pre-0-1.0 00000100:00080000:1.0:1776189323.971600:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189323.971603:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189323.971606:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b45ebec6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664326592:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189323.971675:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003b5d789a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664326656:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189323.971686:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c5b17f98 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664327040:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189323.971697:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c3b654b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664326848:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189323.971701:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664326720 00000100:00100000:2.0:1776189323.971750:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57333 00000100:00100000:2.0:1776189323.971754:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003f21295b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664326720:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189323.971763:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189323.971912:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007fd39f2e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664326976:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189323.976410:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007fd39f2e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664326976:12345-0@lo:400:kworker/u8:0.0 Request processed in 6623us (6982us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189323.976426:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9906 00000100:00100000:0.0:1776189323.977761:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e310c888 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664326976:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189323.982270:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003f21295b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664326720:12345-0@lo:38:kworker/u8:0.0 Request processed in 10507us (12381us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189323.982286:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57333 00000100:00080000:1.0:1776189323.982570:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189323.982722:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189323.982729:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cccdb2ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664326720:0@lo:38:kworker/u8:0.0 00010000:02020000:2.0:1776189323.988777:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189324.245389:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189324.284481:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189324.500275:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189324.756286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189325.012329:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189325.271995:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189325.308776:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189325.524542:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189325.780351:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189326.036352:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189326.292311:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189326.332460:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189326.549445:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189326.804277:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189327.060587:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189327.316399:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189327.356387:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189327.572300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189327.738965:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189327.739016:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189327.739089:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654951424 00000100:00100000:1.0:1776189327.739103:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189327.739142:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654951488 00000100:00100000:3.0:1776189327.739206:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9908 00000100:00100000:3.0:1776189327.739214:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d6111a35 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654951424:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189327.739221:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57334 00000100:00100000:0.0:1776189327.739227:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a12b3db8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654951488:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776189327.739244:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776189327.739298:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d6111a35 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654951424:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 84us (281us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189327.739305:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9908 00000100:00100000:0.0:1776189327.739746:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a12b3db8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654951488:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 517us (642us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189327.739757:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57334 00000100:00100000:2.0:1776189327.740631:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654951360 00000100:00100000:2.0:1776189327.740692:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1432 00000100:00100000:2.0:1776189327.740697:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000002b5a0b3 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654951360:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189327.740746:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000002b5a0b3 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654951360:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 48us (1793us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189327.740753:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1432 00010000:02020000:2.0:1776189327.828779:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189328.061867:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a4221496 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664327168:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189328.061935:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189328.062076:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664327168 00000100:00100000:3.0:1776189328.062124:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6748 00000100:00100000:3.0:1776189328.062130:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000932debd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664327168:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189328.062162:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189328.062230:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000932debd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664327168:12345-0@lo:13:osp-pre-0-0.0 Request processed in 99us (295us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189328.062237:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6748 00000100:00100000:3.0:1776189328.062281:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a4221496 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664327168:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189328.062489:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b0629a6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664327232:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189328.062522:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189328.062594:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664327232 00000100:00100000:1.0:1776189328.062626:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6749 00000100:00100000:1.0:1776189328.062630:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001aa39bf8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664327232:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189328.062646:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189328.062679:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001aa39bf8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664327232:12345-0@lo:13:osp-pre-1-0.0 Request processed in 49us (157us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189328.062685:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6749 00000100:00100000:1.0:1776189328.062713:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b0629a6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664327232:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189328.084295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189328.340638:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189328.381005:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189328.596277:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189328.852274:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189329.084652:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000aecac7d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664327296:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189329.084695:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a4221496 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664327360:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189329.084740:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1776189329.084780:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189329.084786:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189329.084789:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189329.084793:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9470 00000100:00080000:3.0:1776189329.084795:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189329.084800:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189329.084831:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c68cc49e x1862459664327552/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189329.084857:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189329.084860:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189329.084862:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189329.084866:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9470 00000100:00080000:3.0:1776189329.084868:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189329.084871:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189329.084880:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c3d6c201 x1862459664327680/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189329.084886:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189329.084889:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189329.084890:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189329.084893:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9470 00000100:00080000:3.0:1776189329.084895:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189329.084897:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189329.084906:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008c04acb9 x1862459664327744/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189329.084966:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052e42ae8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664327488:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189329.084995:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189329.085030:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000068da6036 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664327936:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189329.085052:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189329.085075:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1776189329.085080:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a45b748c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664327424:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189329.085107:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189329.085127:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008206d902 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664327872:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189329.085150:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189329.085198:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c68cc49e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664327552:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189329.085223:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189329.085239:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3d6c201 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664327680:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189329.085261:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664327424 00000100:00100000:3.0:1776189329.085264:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189329.085291:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c04acb9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664327744:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189329.085307:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1433 00000100:00100000:2.0:1776189329.085314:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000064f5974 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664327424:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189329.085320:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189329.085393:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000064f5974 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664327424:12345-0@lo:400:kworker/u8:0.0 Request processed in 79us (285us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189329.085433:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664327488 00000100:00100000:3.0:1776189329.085462:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664327936 00000100:00100000:3.0:1776189329.085479:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664327552 00000100:00100000:3.0:1776189329.085488:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664327680 00000100:00100000:3.0:1776189329.085504:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9909 00000100:00100000:3.0:1776189329.085507:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000038a7e045 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664327488:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189329.085539:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000038a7e045 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664327488:12345-0@lo:400:kworker/u8:0.0 Request processed in 31us (544us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189329.085546:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9909 00000100:00100000:3.0:1776189329.085563:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9910 00000100:00100000:3.0:1776189329.085566:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000032d9de2c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664327936:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189329.085591:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000032d9de2c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664327936:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (539us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189329.085595:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9910 00000100:00100000:3.0:1776189329.085606:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57335 00000100:00100000:3.0:1776189329.085610:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003cd25a35 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664327552:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189329.085623:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189329.085653:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003cd25a35 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664327552:12345-0@lo:38:kworker/u8:0.0 Request processed in 41us (429us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189329.085659:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57335 00000100:00100000:3.0:1776189329.085669:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57336 00000100:00100000:3.0:1776189329.085672:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007bd06d4b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664327680:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189329.085796:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189329.085816:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007bd06d4b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664327680:12345-0@lo:38:kworker/u8:0.0 Request processed in 143us (554us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189329.085820:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57336 00000100:00100000:3.0:1776189329.085835:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664327744 00000100:00100000:3.0:1776189329.085853:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9111 00000100:00100000:3.0:1776189329.085856:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000032656100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664327744:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189329.085860:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189329.085924:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000032656100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664327744:12345-0@lo:38:kworker/u8:0.0 Request processed in 66us (608us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189329.085931:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9111 00000100:00080000:3.0:1776189329.085993:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189329.085997:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189329.086000:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c68cc49e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664327552:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776189329.086008:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189329.086010:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189329.086012:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3d6c201 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664327680:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776189329.086018:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189329.086020:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189329.086022:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c04acb9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664327744:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189329.086075:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a45b748c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664327424:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189329.086086:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052e42ae8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664327488:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189329.086094:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000068da6036 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664327936:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189329.086230:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664327872 00000100:00100000:3.0:1776189329.086257:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8522 00000100:00100000:3.0:1776189329.086261:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e532ce7c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664327872:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189329.086295:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e532ce7c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664327872:12345-0@lo:400:kworker/u8:0.0 Request processed in 33us (1144us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189329.086301:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8522 00000100:00100000:3.0:1776189329.086342:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008206d902 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664327872:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189329.086543:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1433 00000100:00100000:1.0:1776189329.087594:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664327296 00000100:00100000:1.0:1776189329.087656:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6750 00000100:00100000:1.0:1776189329.087662:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003a31bce8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664327296:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189329.087684:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189329.087737:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003a31bce8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664327296:12345-0@lo:13:osp-pre-0-1.0 Request processed in 75us (3000us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189329.087744:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6750 00000100:00100000:0.0:1776189329.088733:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000aecac7d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664327296:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189329.088769:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000002d6a3fa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664327616:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189329.088813:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189329.088837:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006302d616 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664328000:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189329.088857:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189329.088866:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189329.088879:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664327616 00000100:00100000:0.0:1776189329.088887:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1776189329.088894:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed29900a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664327808:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189329.088921:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189329.088933:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9911 00000100:00100000:3.0:1776189329.088939:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000aecac7d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664327616:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189329.088946:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000838deece pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664328064:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189329.088970:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664328000 00000100:00100000:0.0:1776189329.088984:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189329.089004:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664327808 00000100:00100000:3.0:1776189329.089004:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000aecac7d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664327616:12345-0@lo:400:kworker/u8:0.0 Request processed in 63us (190us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189329.089014:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9911 00000100:00100000:1.0:1776189329.089026:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664328064 00000100:00100000:1.0:1776189329.089054:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8523 00000100:00100000:1.0:1776189329.089060:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007ca965f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664328000:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189329.089065:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8524 00000100:00100000:3.0:1776189329.089072:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008be8e55e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664327808:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189329.089081:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000002d6a3fa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664327616:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189329.089110:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007ca965f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664328000:12345-0@lo:400:kworker/u8:0.0 Request processed in 49us (246us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189329.089117:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008be8e55e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664327808:12345-0@lo:400:kworker/u8:0.0 Request processed in 44us (195us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189329.089119:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8523 00000100:00100000:3.0:1776189329.089128:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8524 00000100:00100000:0.0:1776189329.089137:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed29900a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664327808:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189329.089143:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8525 00000100:00100000:1.0:1776189329.089149:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b25bee14 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664328064:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189329.089153:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006302d616 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664328000:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189329.089218:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b25bee14 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664328064:12345-0@lo:400:kworker/u8:0.0 Request processed in 67us (234us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189329.089225:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8525 00000100:00100000:0.0:1776189329.089229:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000838deece pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664328064:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189329.089905:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664327360 00000100:00100000:2.0:1776189329.089947:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6751 00000100:00100000:2.0:1776189329.089951:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005929d549 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664327360:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189329.089967:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189329.090050:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005929d549 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664327360:12345-0@lo:13:osp-pre-1-1.0 Request processed in 98us (1195us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189329.090057:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6751 00000100:00100000:1.0:1776189329.090151:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a4221496 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664327360:0@lo:13:osp-pre-1-1.0 00010000:02020000:2.0:1776189329.109367:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189329.364282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189329.406632:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189329.620277:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189329.876768:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189330.132290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189330.388745:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189330.428671:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189330.644640:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189330.900305:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189331.156294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189331.412361:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189331.452750:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189331.668294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189331.924294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189332.180305:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189332.436319:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189332.478404:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189332.693118:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189332.858700:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189332.858750:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189332.858781:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189332.858932:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654951680 00000100:00100000:2.0:1776189332.859046:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654951552 00000100:00100000:2.0:1776189332.859089:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1434 00000100:00100000:2.0:1776189332.859096:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000084b636aa pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654951552:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776189332.859211:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000084b636aa pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654951552:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 75us (481us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189332.859219:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1434 00000100:00100000:3.0:1776189332.859399:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654951616 00000100:00100000:1.0:1776189332.859508:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9912 00000100:00100000:1.0:1776189332.859514:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003a195883 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654951616:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776189332.859566:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003a195883 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654951616:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 51us (784us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189332.859573:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9912 00000100:00100000:1.0:1776189332.859588:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57337 00000100:00100000:1.0:1776189332.859592:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003179aec9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654951680:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:1.0:1776189332.859608:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189332.859635:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003179aec9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654951680:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 42us (885us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189332.859641:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57337 00010000:02020000:2.0:1776189332.948446:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189333.180528:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b0629a6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664328192:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189333.180604:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189333.180637:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee307f96 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664328128:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189333.180659:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189333.180951:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664328192 00000100:00100000:3.0:1776189333.181009:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664328128 00000100:00100000:3.0:1776189333.181032:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6752 00000100:00100000:3.0:1776189333.181038:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008a791fe4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664328192:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776189333.181067:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189333.181142:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6753 00000100:00100000:2.0:1776189333.181149:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fd1686e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664328128:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189333.181166:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189333.184372:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fd1686e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664328128:12345-0@lo:13:osp-pre-0-0.0 Request processed in 3214us (3704us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189333.184386:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6753 00000100:00100000:3.0:1776189333.188776:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008a791fe4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664328192:12345-0@lo:13:osp-pre-1-0.0 Request processed in 7728us (8164us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189333.188793:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6752 00000100:00100000:0.0:1776189333.194701:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b0629a6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664328192:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189333.194731:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee307f96 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664328128:0@lo:13:osp-pre-0-0.0 00010000:02020000:2.0:1776189333.205942:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189333.461244:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189333.500680:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189333.716522:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189333.972294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189334.204589:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000027d4ac99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664328256:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189334.204659:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189334.204753:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664328256 00000100:00100000:1.0:1776189334.204798:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6754 00000100:00100000:1.0:1776189334.204803:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cccdb2ad pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664328256:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189334.204825:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189334.204873:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cccdb2ad pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664328256:12345-0@lo:13:osp-pre-1-1.0 Request processed in 70us (215us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189334.204880:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6754 00000100:00100000:3.0:1776189334.205015:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000027d4ac99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664328256:0@lo:13:osp-pre-1-1.0 00000100:00080000:0.0:1776189334.205273:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189334.205277:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189334.205280:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189334.205284:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9476 00000100:00080000:0.0:1776189334.205287:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776189334.205290:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189334.205308:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b9f0254e x1862459664328448/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189334.205337:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189334.205340:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189334.205341:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189334.205344:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9476 00000100:00080000:0.0:1776189334.205346:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776189334.205348:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189334.205358:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000006c1656a x1862459664328576/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189334.205364:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189334.205369:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189334.205370:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189334.205373:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9476 00000100:00080000:0.0:1776189334.205375:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776189334.205377:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189334.205387:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005f3b872c x1862459664328640/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189334.205480:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8bc9a84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664328512:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189334.205517:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189334.205537:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6402bd2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664328896:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189334.205558:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189334.205576:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776189334.205582:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000053ddeb10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664328704:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189334.205603:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189334.205639:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664328512 00000100:00100000:0.0:1776189334.205667:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9913 00000100:00100000:0.0:1776189334.205670:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd47b990 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664328512:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189334.205838:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002c2b2af4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664328384:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189334.205863:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189334.205878:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007c812403 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664328832:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189334.205895:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189334.205908:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1776189334.205912:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000087513f3f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664328320:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189334.205935:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189334.205946:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e8ddaf27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664328768:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189334.205962:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189334.205982:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b9f0254e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664328448:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189334.206002:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189334.206015:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000006c1656a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664328576:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189334.206033:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189334.206043:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005f3b872c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664328640:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189334.206061:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189334.206085:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664328896 00000100:00100000:3.0:1776189334.206108:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664328704 00000100:00100000:3.0:1776189334.206121:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664328768 00000100:00100000:3.0:1776189334.206136:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8526 00000100:00100000:3.0:1776189334.206140:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007c231a0e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664328896:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189334.206168:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007c231a0e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664328896:12345-0@lo:400:kworker/u8:0.0 Request processed in 27us (609us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189334.206193:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8526 00000100:00100000:3.0:1776189334.206205:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8527 00000100:00100000:3.0:1776189334.206209:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000034c47c21 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664328704:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189334.206245:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000034c47c21 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664328704:12345-0@lo:400:kworker/u8:0.0 Request processed in 35us (642us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189334.206252:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8527 00000100:00100000:3.0:1776189334.206261:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8528 00000100:00100000:3.0:1776189334.206264:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000098a9e6f8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664328768:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189334.206289:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000098a9e6f8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664328768:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (327us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189334.206295:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8528 00000100:00100000:3.0:1776189334.206313:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664328384 00000100:00100000:3.0:1776189334.206326:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664328832 00000100:00100000:3.0:1776189334.206334:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664328448 00000100:00100000:3.0:1776189334.206341:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664328576 00000100:00100000:3.0:1776189334.206349:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9914 00000100:00100000:3.0:1776189334.206352:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000027d4ac99 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664328384:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189334.206376:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000027d4ac99 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664328384:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (513us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189334.206381:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9914 00000100:00100000:3.0:1776189334.206391:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9915 00000100:00100000:3.0:1776189334.206394:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008e312a63 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664328832:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189334.206414:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008e312a63 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664328832:12345-0@lo:400:kworker/u8:0.0 Request processed in 19us (520us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189334.206419:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9915 00000100:00100000:3.0:1776189334.206428:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57338 00000100:00100000:3.0:1776189334.206431:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ee307f96 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664328448:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189334.206444:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189334.206471:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ee307f96 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664328448:12345-0@lo:38:kworker/u8:0.0 Request processed in 39us (468us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189334.206477:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57338 00000100:00100000:3.0:1776189334.206486:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57339 00000100:00100000:3.0:1776189334.206489:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a4221496 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664328576:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189334.206495:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189334.206514:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a4221496 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664328576:12345-0@lo:38:kworker/u8:0.0 Request processed in 24us (481us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189334.206519:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57339 00000100:00100000:3.0:1776189334.206547:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664328640 00000100:00100000:3.0:1776189334.206564:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9112 00000100:00100000:3.0:1776189334.206567:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000838deece pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664328640:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189334.206574:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189334.206596:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000838deece pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664328640:12345-0@lo:38:kworker/u8:0.0 Request processed in 28us (535us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189334.206601:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9112 00000100:00080000:3.0:1776189334.206641:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189334.206644:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189334.206648:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b9f0254e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664328448:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776189334.206655:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189334.206658:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189334.206660:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000006c1656a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664328576:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776189334.206667:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189334.206669:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189334.206671:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f3b872c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664328640:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189334.206772:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e8ddaf27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664328768:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189334.206784:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002c2b2af4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664328384:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189334.206793:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007c812403 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664328832:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189334.206860:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000053ddeb10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664328704:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189334.206871:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8bc9a84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664328512:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189334.206880:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6402bd2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664328896:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189334.207096:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776189334.207105:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003abc2764 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664328960:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189334.207136:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189334.207236:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664328320 00000100:00100000:2.0:1776189334.207278:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1435 00000100:00100000:2.0:1776189334.207282:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c1f77e46 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664328320:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189334.207321:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c1f77e46 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664328320:12345-0@lo:400:kworker/u8:0.0 Request processed in 38us (1386us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189334.207327:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1435 00000100:00100000:2.0:1776189334.207359:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000087513f3f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664328320:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189334.207378:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664328960 00000100:00100000:2.0:1776189334.207400:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6755 00000100:00100000:2.0:1776189334.207403:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c6bc9cdf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664328960:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189334.207416:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189334.207444:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c6bc9cdf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664328960:12345-0@lo:13:osp-pre-0-1.0 Request processed in 40us (308us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189334.207449:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6755 00000100:00100000:0.0:1776189334.207525:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd47b990 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664328512:12345-0@lo:400:kworker/u8:0.0 Request processed in 1853us (2007us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189334.207533:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9913 00000100:00100000:1.0:1776189334.207601:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003abc2764 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664328960:0@lo:13:osp-pre-0-1.0 00010000:02020000:2.0:1776189334.229880:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189334.485318:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189334.532076:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189334.740293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189334.996301:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189335.253106:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189335.508521:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189335.549219:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189335.764299:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189336.021009:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189336.277588:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189336.532294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189336.573243:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189336.788408:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189337.044294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189337.300288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189337.556925:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189337.596676:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189337.812283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189337.981855:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189337.982397:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189337.982420:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654951744 00000100:00100000:1.0:1776189337.982545:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1436 00000100:00100000:1.0:1776189337.982554:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f055c03f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654951744:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189337.982564:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189337.982648:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f055c03f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654951744:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 95us (806us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189337.982658:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1436 00000100:00100000:0.0:1776189337.985763:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654951808 00000100:00100000:1.0:1776189337.985824:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654951872 00000100:00100000:0.0:1776189337.985853:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9916 00000100:00100000:1.0:1776189337.985864:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57340 00000100:00100000:0.0:1776189337.985865:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006302d616 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654951808:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189337.985873:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ed29900a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654951872:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776189337.985895:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189337.985963:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ed29900a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654951872:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 91us (3400us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189337.985971:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57340 00000100:00100000:0.0:1776189337.986147:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006302d616 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654951808:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 101us (3568us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189337.986161:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9916 00010000:02020000:2.0:1776189338.068618:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189338.301949:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776189338.301985:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000002d6a3fa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664329024:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189338.302094:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189338.302223:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664329024 00000100:00100000:1.0:1776189338.302392:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e310c888 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664329088:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189338.302421:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189338.302507:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664329088 00000100:00100000:3.0:1776189338.302561:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6756 00000100:00100000:3.0:1776189338.302567:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005cdf5f34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664329088:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776189338.302607:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189338.302658:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005cdf5f34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664329088:12345-0@lo:13:osp-pre-1-0.0 Request processed in 90us (237us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189338.302666:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6756 00000100:00100000:1.0:1776189338.302774:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e310c888 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664329088:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189338.303212:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6757 00000100:00100000:2.0:1776189338.303218:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005cd76944 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664329024:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189338.303241:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189338.304087:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005cd76944 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664329024:12345-0@lo:13:osp-pre-0-0.0 Request processed in 862us (1989us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189338.304100:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6757 00000100:00100000:3.0:1776189338.308992:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000002d6a3fa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664329024:0@lo:13:osp-pre-0-0.0 00010000:02020000:2.0:1776189338.324345:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189338.580345:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189338.620455:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189338.836284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189339.092738:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189339.325566:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776189339.325591:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d4382e95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664329152:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189339.325679:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189339.325999:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664329152 00000100:00100000:1.0:1776189339.326051:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6758 00000100:00100000:1.0:1776189339.326057:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000002d6a3fa pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664329152:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189339.326081:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189339.326283:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000002d6a3fa pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664329152:12345-0@lo:13:osp-pre-0-1.0 Request processed in 225us (605us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189339.326290:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6758 00000100:00100000:1.0:1776189339.326338:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d4382e95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664329152:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189339.327005:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776189339.327020:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a7b8e04c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664329216:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189339.327061:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1776189339.327146:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189339.327150:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189339.327153:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189339.327157:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9481 00000100:00080000:2.0:1776189339.327160:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00080000:2.0:1776189339.327164:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189339.327219:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b45ebec6 x1862459664329408/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189339.327243:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189339.327247:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189339.327248:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189339.327252:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9481 00000100:00080000:2.0:1776189339.327254:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00080000:2.0:1776189339.327257:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189339.327273:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000739e313a x1862459664329536/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189339.327280:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189339.327282:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189339.327284:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189339.327286:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9481 00000100:00080000:2.0:1776189339.327288:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 6s 00000100:00080000:2.0:1776189339.327291:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189339.327298:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b385e94b x1862459664329600/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189339.327345:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c5b17f98 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664329280:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189339.327374:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189339.327389:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009015c790 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664329728:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189339.327408:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189339.327423:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1776189339.327427:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003b5d789a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664329344:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189339.327451:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189339.327466:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e310c888 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664329792:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189339.327485:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189339.327508:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b45ebec6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664329408:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189339.327528:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189339.327542:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000739e313a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664329536:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189339.327560:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189339.327571:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b385e94b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664329600:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189339.327592:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189339.327637:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664329728 00000100:00100000:2.0:1776189339.327682:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8529 00000100:00100000:2.0:1776189339.327687:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009b0629a6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664329728:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189339.327988:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009b0629a6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664329728:12345-0@lo:400:kworker/u8:1.0 Request processed in 300us (578us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189339.327997:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8529 00000100:00100000:2.0:1776189339.328022:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664329344 00000100:00100000:2.0:1776189339.328044:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664329792 00000100:00100000:2.0:1776189339.328059:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664329408 00000100:00100000:2.0:1776189339.328066:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664329536 00000100:00100000:2.0:1776189339.328073:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9917 00000100:00100000:2.0:1776189339.328077:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000040b53b34 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664329344:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189339.328116:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000040b53b34 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664329344:12345-0@lo:400:kworker/u8:1.0 Request processed in 26us (653us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189339.328123:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9917 00000100:00100000:2.0:1776189339.328134:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9918 00000100:00100000:2.0:1776189339.328137:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000094ac8433 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664329792:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189339.328161:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000094ac8433 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664329792:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (675us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189339.328166:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9918 00000100:00100000:2.0:1776189339.328199:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57341 00000100:00100000:2.0:1776189339.328202:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000673b41e8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664329408:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189339.328226:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189339.328256:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000673b41e8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664329408:12345-0@lo:38:kworker/u8:1.0 Request processed in 53us (727us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189339.328261:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57341 00000100:00100000:2.0:1776189339.328270:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57342 00000100:00100000:2.0:1776189339.328273:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e139d197 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664329536:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189339.328278:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189339.328294:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e139d197 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664329536:12345-0@lo:38:kworker/u8:1.0 Request processed in 21us (735us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189339.328299:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57342 00000100:00100000:2.0:1776189339.328327:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664329600 00000100:00100000:2.0:1776189339.328344:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9113 00000100:00100000:2.0:1776189339.328347:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003fb63a1c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664329600:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189339.328354:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189339.328376:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003fb63a1c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664329600:12345-0@lo:38:kworker/u8:1.0 Request processed in 28us (784us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189339.328381:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9113 00000100:00100000:2.0:1776189339.328420:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664329280 00000100:00100000:2.0:1776189339.328443:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1437 00000100:00100000:2.0:1776189339.328446:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000353f317b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664329280:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189339.328473:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000353f317b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664329280:12345-0@lo:400:kworker/u8:1.0 Request processed in 26us (1100us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189339.328479:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1437 00000100:00100000:2.0:1776189339.328508:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a0146fc7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664329472:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189339.328532:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189339.328585:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b9092100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664329664:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189339.328626:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189339.328657:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664329664 00000100:00100000:1.0:1776189339.328680:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8530 00000100:00100000:1.0:1776189339.328685:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d4382e95 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664329664:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189339.328733:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d4382e95 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664329664:12345-0@lo:400:kworker/u8:1.0 Request processed in 47us (107us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189339.328738:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8530 00000100:00100000:1.0:1776189339.328774:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b9092100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664329664:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189339.329407:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664329216 00000100:00100000:0.0:1776189339.329442:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6759 00000100:00100000:0.0:1776189339.329447:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006c3b654b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664329216:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189339.329464:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189339.329502:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006c3b654b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664329216:12345-0@lo:13:osp-pre-1-1.0 Request processed in 54us (2441us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189339.329507:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6759 00000100:00080000:1.0:1776189339.331334:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189339.331344:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189339.331350:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b45ebec6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664329408:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776189339.331361:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189339.331363:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189339.331366:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000739e313a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664329536:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776189339.331375:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189339.331378:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189339.331380:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b385e94b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664329600:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189339.331425:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003b5d789a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664329344:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189339.331436:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e310c888 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664329792:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189339.331445:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c5b17f98 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664329280:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189339.331454:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009015c790 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664329728:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189339.342930:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a7b8e04c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664329216:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189339.343925:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664329472 00000100:00100000:0.0:1776189339.343998:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9919 00000100:00100000:0.0:1776189339.344003:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003d6663c8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664329472:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189339.344165:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003d6663c8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664329472:12345-0@lo:400:kworker/u8:1.0 Request processed in 160us (15632us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189339.344213:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9919 00010000:02020000:2.0:1776189339.348258:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189339.348448:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a0146fc7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664329472:0@lo:400:kworker/u8:1.0 00010000:02020000:2.0:1776189339.604545:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189339.645542:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189339.860278:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189340.116308:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189340.375946:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189340.630144:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189340.669686:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189340.884282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189341.141569:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189341.396306:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189341.652286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189341.692496:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189341.908291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189342.164773:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189342.420295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189342.677711:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189342.716370:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189342.932280:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189343.099538:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189343.099599:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189343.099629:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654951936 00000100:00100000:2.0:1776189343.099640:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189343.099683:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1438 00000100:00100000:0.0:1776189343.099689:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003c5b81e6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654951936:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189343.099714:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654952064 00000100:00100000:1.0:1776189343.104897:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654952000 00000100:00100000:1.0:1776189343.104964:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9920 00000100:00100000:1.0:1776189343.104970:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f078b3a8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654952000:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189343.105123:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57343 00000100:00100000:2.0:1776189343.105129:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fe93ff8e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654952064:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776189343.105145:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189343.108834:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003c5b81e6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654951936:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 9135us (9296us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189343.108850:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1438 00000100:00100000:1.0:1776189343.109970:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f078b3a8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654952000:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 4994us (10326us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189343.109983:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9920 00000100:00100000:2.0:1776189343.111469:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fe93ff8e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654952064:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 6333us (11865us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189343.111483:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57343 00010000:02020000:2.0:1776189343.188315:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189343.424905:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776189343.424933:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008206d902 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664329856:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189343.425007:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189343.425145:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664329856 00000100:00100000:1.0:1776189343.425212:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6760 00000100:00100000:1.0:1776189343.425217:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009b864aa6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664329856:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189343.425245:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189343.425284:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009b864aa6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664329856:12345-0@lo:13:osp-pre-0-0.0 Request processed in 66us (278us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189343.425291:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6760 00000100:00100000:1.0:1776189343.425335:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008206d902 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664329856:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189343.425459:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000068da6036 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664329920:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189343.425496:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189343.425707:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664329920 00000100:00100000:3.0:1776189343.425740:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6761 00000100:00100000:3.0:1776189343.425744:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000327439e5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664329920:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776189343.425761:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189343.426056:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000068da6036 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664329920:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189343.426128:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000327439e5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664329920:12345-0@lo:13:osp-pre-1-0.0 Request processed in 55us (304us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189343.426137:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6761 00010000:02020000:2.0:1776189343.444590:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189343.700267:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189343.740395:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189343.956273:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189344.212278:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:3.0:1776189344.444485:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189344.444492:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189344.444495:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189344.444503:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9486 00000100:00080000:3.0:1776189344.444507:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00080000:3.0:1776189344.444512:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189344.444529:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f8dcf939 x1862459664330112/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189344.444554:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189344.444557:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189344.444558:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189344.444561:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9486 00000100:00080000:3.0:1776189344.444563:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:3.0:1776189344.444566:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189344.444576:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000052e42ae8 x1862459664330240/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189344.444583:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189344.444586:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189344.444588:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189344.444590:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9486 00000100:00080000:3.0:1776189344.444592:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 11s 00000100:00080000:3.0:1776189344.444595:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189344.444605:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a45b748c x1862459664330304/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189344.444741:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c27afb9e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664329984:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189344.444821:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189344.444844:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3d6c201 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664330432:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189344.444869:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189344.444892:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1776189344.444897:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000efff0bc3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664330048:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189344.444919:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189344.444931:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c68cc49e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664330496:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189344.444950:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189344.444973:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f8dcf939 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664330112:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189344.445005:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189344.445020:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052e42ae8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664330240:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189344.445038:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189344.445050:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a45b748c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664330304:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189344.445068:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189344.445243:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024c95b46 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664330688:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189344.445269:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189344.445301:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664330688 00000100:00100000:3.0:1776189344.445336:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6762 00000100:00100000:3.0:1776189344.445341:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000887d8a72 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664330688:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776189344.445361:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189344.445402:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000887d8a72 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664330688:12345-0@lo:13:osp-pre-0-1.0 Request processed in 60us (132us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189344.445409:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6762 00000100:00100000:3.0:1776189344.445452:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024c95b46 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664330688:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189344.445495:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664330048 00000100:00100000:3.0:1776189344.445518:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664330496 00000100:00100000:3.0:1776189344.445531:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664330112 00000100:00100000:3.0:1776189344.445538:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664330240 00000100:00100000:3.0:1776189344.445547:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9921 00000100:00100000:3.0:1776189344.445550:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c6bc9cdf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664330048:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189344.445587:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c6bc9cdf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664330048:12345-0@lo:400:kworker/u8:1.0 Request processed in 36us (668us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189344.445592:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9921 00000100:00100000:3.0:1776189344.445604:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9922 00000100:00100000:3.0:1776189344.445607:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005cdf5f34 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664330496:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189344.445632:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005cdf5f34 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664330496:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (682us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189344.445637:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9922 00000100:00100000:3.0:1776189344.445647:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57344 00000100:00100000:3.0:1776189344.445650:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005cd76944 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664330112:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189344.445663:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189344.445693:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005cd76944 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664330112:12345-0@lo:38:kworker/u8:1.0 Request processed in 43us (699us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189344.445699:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57344 00000100:00100000:3.0:1776189344.445709:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57345 00000100:00100000:3.0:1776189344.445711:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009b864aa6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664330240:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189344.445717:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189344.445736:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009b864aa6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664330240:12345-0@lo:38:kworker/u8:1.0 Request processed in 24us (697us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189344.445740:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57345 00000100:00100000:3.0:1776189344.445759:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664330432 00000100:00100000:3.0:1776189344.445781:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8531 00000100:00100000:3.0:1776189344.445784:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cc4b762b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664330432:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189344.445810:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cc4b762b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664330432:12345-0@lo:400:kworker/u8:1.0 Request processed in 25us (941us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189344.445818:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8531 00000100:00100000:3.0:1776189344.445834:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c04acb9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664330368:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189344.445855:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189344.445867:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008206d902 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664330624:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189344.445884:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189344.445897:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1776189344.445903:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000002d6a3fa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664330176:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189344.445922:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189344.445934:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000068da6036 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664330560:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189344.445952:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189344.445974:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664330368 00000100:00100000:3.0:1776189344.445992:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664330624 00000100:00100000:3.0:1776189344.446021:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664330560 00000100:00100000:3.0:1776189344.446036:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8532 00000100:00100000:3.0:1776189344.446039:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000024c95b46 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664330368:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189344.446069:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000024c95b46 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664330368:12345-0@lo:400:kworker/u8:1.0 Request processed in 28us (214us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189344.446073:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8532 00000100:00100000:3.0:1776189344.446083:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8533 00000100:00100000:3.0:1776189344.446086:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba2818a5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664330624:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189344.446504:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba2818a5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664330624:12345-0@lo:400:kworker/u8:1.0 Request processed in 415us (618us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189344.446510:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8533 00000100:00100000:3.0:1776189344.446523:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8534 00000100:00100000:3.0:1776189344.446527:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001b0d11fc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664330560:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189344.446558:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001b0d11fc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664330560:12345-0@lo:400:kworker/u8:1.0 Request processed in 31us (606us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189344.446563:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8534 00000100:00100000:3.0:1776189344.446608:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000068da6036 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664330560:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189344.446616:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c04acb9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664330368:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189344.446623:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008206d902 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664330624:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189344.446649:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664330176 00000100:00100000:3.0:1776189344.446667:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9923 00000100:00100000:3.0:1776189344.446670:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cf6b5d65 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664330176:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189344.446701:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cf6b5d65 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664330176:12345-0@lo:400:kworker/u8:1.0 Request processed in 30us (779us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189344.446707:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9923 00000100:00100000:3.0:1776189344.446729:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000002d6a3fa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664330176:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189344.446753:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000efff0bc3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664330048:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189344.446763:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c68cc49e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664330496:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189344.446773:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3d6c201 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664330432:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189344.446841:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664330304 00000100:00100000:3.0:1776189344.446861:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9114 00000100:00100000:3.0:1776189344.446864:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000bfa9efc pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664330304:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189344.446874:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189344.446901:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000bfa9efc pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664330304:12345-0@lo:38:kworker/u8:1.0 Request processed in 35us (1832us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189344.446911:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9114 00000100:00080000:3.0:1776189344.446951:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189344.446955:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189344.446958:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f8dcf939 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664330112:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776189344.446969:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189344.446972:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189344.446974:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052e42ae8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664330240:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776189344.446982:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189344.446984:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189344.446986:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a45b748c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664330304:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189344.447542:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664329984 00000100:00100000:0.0:1776189344.447578:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1439 00000100:00100000:0.0:1776189344.447582:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008e1954f2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664329984:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189344.447620:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008e1954f2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664329984:12345-0@lo:400:kworker/u8:1.0 Request processed in 37us (2801us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189344.447626:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1439 00000100:00100000:2.0:1776189344.447699:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc7962a4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664330752:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189344.447734:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189344.447818:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664330752 00000100:00100000:0.0:1776189344.447851:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6763 00000100:00100000:0.0:1776189344.447854:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f09d90e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664330752:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189344.447868:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189344.447899:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f09d90e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664330752:12345-0@lo:13:osp-pre-1-1.0 Request processed in 44us (166us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189344.447905:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6763 00000100:00100000:2.0:1776189344.448038:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc7962a4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664330752:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189344.448221:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c27afb9e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664329984:0@lo:400:kworker/u8:1.0 00010000:02020000:2.0:1776189344.468544:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189344.724294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189344.765885:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189344.980315:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189345.236286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189345.492617:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189345.749268:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189345.789616:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189346.004599:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189346.260369:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189346.516647:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189346.772325:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189346.812565:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189347.028553:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189347.286392:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189347.541240:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189347.796463:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189347.836497:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189348.052282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189348.220544:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189348.220592:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189348.220737:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189348.220827:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654952128 00000100:00100000:1.0:1776189348.220939:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1440 00000100:00100000:1.0:1776189348.220946:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001aaffad0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654952128:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776189348.221026:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001aaffad0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654952128:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 80us (289us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189348.221044:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1440 00000100:00100000:3.0:1776189348.221670:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654952192 00000100:00100000:3.0:1776189348.222196:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654952256 00000100:00100000:2.0:1776189348.222205:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9924 00000100:00100000:2.0:1776189348.222213:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e42142fc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654952192:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1776189348.222228:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57346 00000100:00100000:3.0:1776189348.222234:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e0bab4ab pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654952256:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:3.0:1776189348.222248:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189348.222264:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e42142fc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654952192:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 50us (1727us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189348.222272:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9924 00000100:00100000:3.0:1776189348.222295:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e0bab4ab pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654952256:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 60us (1702us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189348.222303:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57346 00010000:02020000:2.0:1776189348.308340:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189348.542203:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b5045f9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664330816:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189348.542228:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a543fbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664330880:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189348.542284:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189348.542357:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189348.542595:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664330880 00000100:00100000:1.0:1776189348.542646:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664330816 00000100:00100000:1.0:1776189348.542677:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6764 00000100:00100000:1.0:1776189348.542683:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cccdb2ad pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664330880:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189348.542712:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189348.542754:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cccdb2ad pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664330880:12345-0@lo:13:osp-pre-0-0.0 Request processed in 70us (470us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189348.542761:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6764 00000100:00100000:1.0:1776189348.542778:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6765 00000100:00100000:1.0:1776189348.542782:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c27afb9e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664330816:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189348.542815:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189348.544843:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a543fbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664330880:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189348.545420:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c27afb9e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664330816:12345-0@lo:13:osp-pre-1-0.0 Request processed in 2634us (3061us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189348.545432:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6765 00000100:00100000:3.0:1776189348.546058:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b5045f9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664330816:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189348.564291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189348.820297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189348.860493:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189349.076438:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189349.332287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189349.565914:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776189349.565935:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000030d1626f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664330944:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189349.565998:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189349.566302:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776189349.566310:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc7962a4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664331008:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189349.566340:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189349.566377:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664330944 00000100:00100000:0.0:1776189349.566409:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664331008 00000100:00100000:0.0:1776189349.566421:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6766 00000100:00100000:0.0:1776189349.566426:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007a543fbe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664330944:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189349.566446:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189349.566475:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007a543fbe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664330944:12345-0@lo:13:osp-pre-1-1.0 Request processed in 49us (478us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189349.566482:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6766 00000100:00100000:0.0:1776189349.566496:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6767 00000100:00100000:0.0:1776189349.566499:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a7b8e04c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664331008:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776189349.566507:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189349.566527:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a7b8e04c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664331008:12345-0@lo:13:osp-pre-0-1.0 Request processed in 26us (192us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189349.566535:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6767 00000100:00100000:0.0:1776189349.566573:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc7962a4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664331008:0@lo:13:osp-pre-0-1.0 00000100:00080000:2.0:1776189349.566752:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189349.566756:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189349.566759:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189349.566764:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9491 00000100:00080000:2.0:1776189349.566767:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:2.0:1776189349.566770:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189349.566785:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000733a9e44 x1862459664331200/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189349.566809:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189349.566812:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189349.566813:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189349.566816:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9491 00000100:00080000:2.0:1776189349.566818:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00080000:2.0:1776189349.566821:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189349.566830:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d729b5cd x1862459664331328/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189349.566843:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1776189349.566845:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000008468a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664331136:0@lo:400:kworker/u8:1.0 00000100:00080000:2.0:1776189349.566846:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189349.566847:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189349.566849:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9491 00000100:00080000:2.0:1776189349.566851:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00080000:2.0:1776189349.566854:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189349.566863:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e9c407a2 x1862459664331392/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189349.566876:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189349.566905:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a0c7825a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664331584:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189349.566922:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000748b663e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664331264:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189349.566926:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189349.566940:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1776189349.566944:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae28b57d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664331072:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189349.566946:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189349.566961:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776189349.566966:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000af79f780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664331456:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189349.566967:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189349.566979:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e463b423 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664331520:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189349.566989:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189349.567002:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664331072 00000100:00100000:2.0:1776189349.567014:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664331136 00000100:00100000:1.0:1776189349.567029:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1441 00000100:00100000:1.0:1776189349.567032:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000439c1468 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664331072:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189349.567034:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664331584 00000100:00100000:2.0:1776189349.567046:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664331264 00000100:00100000:2.0:1776189349.567058:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9925 00000100:00100000:1.0:1776189349.567061:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000439c1468 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664331072:12345-0@lo:400:kworker/u8:1.0 Request processed in 28us (94us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189349.567062:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006c3b654b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664331136:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189349.567067:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1441 00000100:00100000:1.0:1776189349.567102:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664331456 00000100:00100000:2.0:1776189349.567102:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006c3b654b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664331136:12345-0@lo:400:kworker/u8:1.0 Request processed in 39us (226us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189349.567108:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9925 00000100:00100000:2.0:1776189349.567118:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9926 00000100:00100000:1.0:1776189349.567120:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8535 00000100:00100000:2.0:1776189349.567122:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0146fc7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664331584:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189349.567123:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000079c74b68 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664331456:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189349.567145:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0146fc7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664331584:12345-0@lo:400:kworker/u8:1.0 Request processed in 23us (219us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189349.567147:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000079c74b68 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664331456:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (159us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189349.567151:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9926 00000100:00100000:1.0:1776189349.567153:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8535 00000100:00100000:2.0:1776189349.567160:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9927 00000100:00100000:2.0:1776189349.567163:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b1d7bdea pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664331264:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189349.567169:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189349.567203:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b1d7bdea pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664331264:12345-0@lo:400:kworker/u8:1.0 Request processed in 39us (258us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189349.567204:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a0c7825a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664331584:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189349.567210:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9927 00000100:00100000:1.0:1776189349.567213:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000008468a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664331136:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189349.567226:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae28b57d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664331072:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189349.567234:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000af79f780 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664331456:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189349.567241:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664331520 00000100:00100000:2.0:1776189349.567242:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000748b663e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664331264:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189349.567260:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8536 00000100:00100000:1.0:1776189349.567263:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009015c790 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664331520:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189349.567311:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009015c790 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664331520:12345-0@lo:400:kworker/u8:1.0 Request processed in 46us (141us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189349.567317:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8536 00000100:00100000:3.0:1776189349.567562:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000733a9e44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664331200:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189349.567592:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189349.567619:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d729b5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664331328:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189349.567639:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189349.567650:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9c407a2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664331392:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189349.567668:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189349.567699:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664331200 00000100:00100000:2.0:1776189349.567721:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664331392 00000100:00100000:3.0:1776189349.567724:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664331328 00000100:00100000:3.0:1776189349.567739:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57347 00000100:00100000:3.0:1776189349.567744:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a45b748c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664331200:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189349.567750:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9115 00000100:00100000:2.0:1776189349.567755:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f8dcf939 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664331392:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189349.567758:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00010000:02000000:2.0:1776189349.567766:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189349.567800:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a45b748c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664331200:12345-0@lo:38:kworker/u8:1.0 Request processed in 55us (205us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189349.567807:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f8dcf939 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664331392:12345-0@lo:38:kworker/u8:1.0 Request processed in 50us (137us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189349.567809:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57347 00000100:00100000:2.0:1776189349.567818:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9115 00000100:00100000:3.0:1776189349.567829:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57348 00000100:00100000:3.0:1776189349.567833:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000052e42ae8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664331328:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189349.567841:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189349.567854:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e463b423 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664331520:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189349.567871:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000052e42ae8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664331328:12345-0@lo:38:kworker/u8:1.0 Request processed in 36us (230us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189349.567877:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57348 00000100:00080000:3.0:1776189349.567937:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189349.567941:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189349.567944:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000733a9e44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664331200:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776189349.567951:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189349.567953:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189349.567954:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d729b5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664331328:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776189349.567961:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189349.567963:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189349.567965:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9c407a2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664331392:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189349.567998:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000030d1626f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664330944:0@lo:13:osp-pre-1-1.0 00010000:02020000:2.0:1776189349.588776:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189349.844559:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189349.884479:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189350.100286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189350.356600:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189350.612271:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189350.868292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189350.908531:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189351.124246:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189351.381979:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189351.636279:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189351.892296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189351.932668:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189352.148282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189352.404318:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189352.660313:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189352.916475:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189352.956533:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189353.172412:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189353.340343:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189353.340435:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189353.343996:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189353.344072:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654952320 00000100:00100000:3.0:1776189353.344136:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1442 00000100:00100000:3.0:1776189353.344143:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006339443e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654952320:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189353.344438:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006339443e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654952320:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 294us (449us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189353.344447:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1442 00000100:00100000:0.0:1776189353.346267:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654952448 00000100:00100000:0.0:1776189353.346399:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654952384 00000100:00100000:0.0:1776189353.346501:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9928 00000100:00100000:0.0:1776189353.346508:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c68cc49e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654952384:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189353.346571:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57349 00000100:00100000:1.0:1776189353.346579:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c3d6c201 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654952448:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00000100:00100000:0.0:1776189353.346581:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c68cc49e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654952384:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 70us (6144us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189353.346591:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9928 00010000:02000000:1.0:1776189353.346601:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189353.346639:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c3d6c201 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654952448:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 61us (6305us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189353.346646:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57349 00010000:02020000:2.0:1776189353.429468:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189353.660402:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000efff0bc3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664331648:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189353.660464:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000002d6a3fa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664331712:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189353.660472:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189353.660553:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664331648 00000100:00100000:1.0:1776189353.660662:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6768 00000100:00100000:1.0:1776189353.660669:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a5008497 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664331648:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189353.660697:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189353.664064:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189353.664332:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664331712 00000100:00100000:3.0:1776189353.664634:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6769 00000100:00100000:3.0:1776189353.664641:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cc7962a4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664331712:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189353.664670:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189353.664733:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cc7962a4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664331712:12345-0@lo:13:osp-pre-0-0.0 Request processed in 91us (675us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189353.664740:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6769 00000100:00100000:0.0:1776189353.665234:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000002d6a3fa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664331712:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189353.667743:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a5008497 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664331648:12345-0@lo:13:osp-pre-1-0.0 Request processed in 7067us (7266us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189353.667760:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6768 00000100:00100000:2.0:1776189353.667881:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000efff0bc3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664331648:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189353.684376:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189353.940965:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189353.983607:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189354.196278:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189354.452343:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189354.686170:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000030d1626f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664331776:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189354.686344:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189354.686428:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008206d902 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664331840:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189354.686452:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1776189354.690154:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189354.690161:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189354.690165:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189354.690200:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9496 00000100:00080000:3.0:1776189354.690204:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189354.690209:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189354.690309:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c04acb9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664331904:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189354.690393:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189354.690418:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000efff0bc3 x1862459664332032/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189354.690459:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189354.690549:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189354.690551:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189354.690555:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9496 00000100:00080000:3.0:1776189354.690557:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189354.690560:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189354.690605:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009b5045f9 x1862459664332160/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189354.690665:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189354.690668:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189354.690670:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189354.690673:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9496 00000100:00080000:3.0:1776189354.690675:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189354.690678:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189354.690703:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c7c1f715 x1862459664332224/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189354.690738:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776189354.690744:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002fbab827 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664332288:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189354.690771:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189354.690800:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f326e309 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664332352:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189354.690821:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189354.690845:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000efff0bc3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664332032:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189354.690851:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000068da6036 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664331968:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189354.690874:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189354.690888:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000911b4242 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664332416:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189354.690903:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189354.690920:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664331904 00000100:00100000:3.0:1776189354.690920:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b5045f9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664332160:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189354.690961:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1443 00000100:00100000:2.0:1776189354.690967:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a3281d23 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664331904:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189354.690978:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189354.690993:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7c1f715 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664332224:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189354.696152:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a3281d23 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664331904:12345-0@lo:400:kworker/u8:1.0 Request processed in 5174us (5750us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189354.696170:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1443 00000100:00100000:2.0:1776189354.696251:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664332288 00000100:00100000:2.0:1776189354.696295:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664332352 00000100:00100000:2.0:1776189354.696323:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8537 00000100:00100000:2.0:1776189354.696328:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000917ca813 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664332288:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189354.697032:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000917ca813 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664332288:12345-0@lo:400:kworker/u8:1.0 Request processed in 701us (6258us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189354.697040:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8537 00000100:00100000:2.0:1776189354.697055:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8538 00000100:00100000:2.0:1776189354.697059:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001ada75b0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664332352:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189354.697089:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001ada75b0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664332352:12345-0@lo:400:kworker/u8:1.0 Request processed in 29us (6268us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189354.697094:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8538 00000100:00100000:2.0:1776189354.697117:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664331968 00000100:00100000:2.0:1776189354.697145:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664332032 00000100:00100000:2.0:1776189354.697160:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664332160 00000100:00100000:2.0:1776189354.697185:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9929 00000100:00100000:1.0:1776189354.697186:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000002d6a3fa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664332096:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189354.697188:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d781d9c8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664331968:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189354.697217:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d781d9c8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664331968:12345-0@lo:400:kworker/u8:1.0 Request processed in 28us (6343us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189354.697220:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189354.697222:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9929 00000100:00100000:1.0:1776189354.697230:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f159c11f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664332480:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189354.697234:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664332096 00000100:00100000:2.0:1776189354.697245:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9930 00000100:00100000:2.0:1776189354.697248:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003b5d789a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664332096:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189354.697262:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189354.697271:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003b5d789a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664332096:12345-0@lo:400:kworker/u8:1.0 Request processed in 23us (52us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189354.697277:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9930 00000100:00100000:2.0:1776189354.697286:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57350 00000100:00100000:2.0:1776189354.697289:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0fb2966 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664332032:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189354.697291:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000002d6a3fa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664332096:0@lo:400:kworker/u8:1.0 00010000:02000000:2.0:1776189354.697301:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189354.697314:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664332480 00000100:00100000:2.0:1776189354.697328:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0fb2966 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664332032:12345-0@lo:38:kworker/u8:1.0 Request processed in 37us (6440us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189354.697333:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57350 00000100:00100000:1.0:1776189354.697334:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8539 00000100:00100000:1.0:1776189354.697337:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b385e94b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664332480:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189354.697343:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57351 00000100:00100000:2.0:1776189354.697345:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cda39ea2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664332160:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189354.697352:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189354.697365:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b385e94b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664332480:12345-0@lo:400:kworker/u8:1.0 Request processed in 27us (115us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189354.697369:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cda39ea2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664332160:12345-0@lo:38:kworker/u8:1.0 Request processed in 23us (6412us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189354.697371:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8539 00000100:00100000:2.0:1776189354.697373:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57351 00000100:00100000:1.0:1776189354.697393:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f159c11f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664332480:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189354.697459:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189354.697499:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664332224 00000100:00100000:3.0:1776189354.697520:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9116 00000100:00100000:3.0:1776189354.697523:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001aa39bf8 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664332224:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189354.697531:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189354.697556:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001aa39bf8 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664332224:12345-0@lo:38:kworker/u8:1.0 Request processed in 32us (97us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189354.697562:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9116 00000100:00080000:3.0:1776189354.697586:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189354.697589:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189354.697593:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000efff0bc3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664332032:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776189354.697601:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189354.697604:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189354.697606:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b5045f9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664332160:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776189354.697614:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189354.697616:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189354.697619:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7c1f715 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664332224:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189354.697660:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189354.697684:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000068da6036 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664331968:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189354.697704:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664332416 00000100:00100000:3.0:1776189354.697724:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9931 00000100:00100000:3.0:1776189354.697728:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003a31bce8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664332416:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189354.697760:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003a31bce8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664332416:12345-0@lo:400:kworker/u8:1.0 Request processed in 31us (100us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189354.697765:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9931 00000100:00100000:3.0:1776189354.697798:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000911b4242 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664332416:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189354.697822:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f326e309 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664332352:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189354.697831:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002fbab827 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664332288:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189354.697839:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c04acb9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664331904:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189354.700221:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664331776 00000100:00100000:0.0:1776189354.700296:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664331840 00000100:00100000:0.0:1776189354.700323:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6770 00000100:00100000:0.0:1776189354.700329:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c5b17f98 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664331776:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189354.700352:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189354.700418:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c5b17f98 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664331776:12345-0@lo:13:osp-pre-1-1.0 Request processed in 90us (14076us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189354.700425:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6770 00000100:00100000:0.0:1776189354.700440:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6771 00000100:00100000:0.0:1776189354.700443:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e310c888 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664331840:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776189354.700453:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189354.700476:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e310c888 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664331840:12345-0@lo:13:osp-pre-0-1.0 Request processed in 33us (14024us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189354.700493:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6771 00000100:00100000:1.0:1776189354.700807:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000030d1626f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664331776:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189354.700823:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008206d902 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664331840:0@lo:13:osp-pre-0-1.0 00010000:02020000:2.0:1776189354.708267:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189354.966164:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189355.004604:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189355.220490:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189355.476284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189355.732370:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189355.990231:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189356.028429:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189356.248142:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189356.500282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189356.756281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189357.012278:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189357.052544:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189357.268280:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189357.524360:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189357.781226:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189358.036978:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189358.076438:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189358.292287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189358.458048:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189358.458102:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189358.458127:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654952512 00000100:00100000:1.0:1776189358.458226:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654952576 00000100:00100000:1.0:1776189358.458274:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9932 00000100:00100000:1.0:1776189358.458282:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fd1686e0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654952576:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776189358.458359:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fd1686e0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654952576:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 76us (256us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189358.458368:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9932 00000100:00100000:0.0:1776189358.458593:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1444 00000100:00100000:0.0:1776189358.458599:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b7303d3d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654952512:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1776189358.458641:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b7303d3d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654952512:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 41us (601us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189358.458647:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1444 00000100:00100000:3.0:1776189358.472622:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189358.472762:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654952640 00000100:00100000:1.0:1776189358.472815:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57352 00000100:00100000:1.0:1776189358.472822:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008a791fe4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654952640:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:1.0:1776189358.472841:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189358.472918:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008a791fe4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654952640:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 95us (304us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189358.472925:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57352 00010000:02020000:2.0:1776189358.548440:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189358.780421:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003258ba66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664332544:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189358.780427:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000087513f3f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664332608:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189358.780487:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189358.780592:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664332608 00000100:00100000:2.0:1776189358.780649:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6772 00000100:00100000:2.0:1776189358.780654:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000030d1626f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664332608:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189358.780682:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189358.780748:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000030d1626f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664332608:12345-0@lo:13:osp-pre-0-0.0 Request processed in 92us (261us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189358.780755:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6772 00000100:00100000:0.0:1776189358.780911:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000087513f3f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664332608:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189358.780975:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189358.782833:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664332544 00000100:00100000:3.0:1776189358.782894:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6773 00000100:00100000:3.0:1776189358.782900:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000739e313a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664332544:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776189358.782928:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189358.784229:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000739e313a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664332544:12345-0@lo:13:osp-pre-1-0.0 Request processed in 1323us (3249us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189358.784243:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6773 00000100:00100000:1.0:1776189358.786289:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003258ba66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664332544:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189358.804919:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189359.060416:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189359.100762:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189359.316399:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189359.572952:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189359.804405:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776189359.804429:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b45ebec6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664332672:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189359.804494:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189359.804544:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bd5316a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664332736:0@lo:13:osp-pre-1-1.0 00000100:00080000:0.0:1776189359.804564:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1776189359.804567:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664332672 00000100:00080000:0.0:1776189359.804569:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189359.804574:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1776189359.804574:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776189359.804582:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9501 00000100:00080000:0.0:1776189359.804585:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1776189359.804591:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189359.804612:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a680c874 x1862459664332928/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189359.804612:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664332736 00000100:00100000:1.0:1776189359.804643:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6774 00000100:00080000:0.0:1776189359.804646:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1776189359.804651:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b9092100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664332672:12345-0@lo:13:osp-pre-0-1.0 00000100:00080000:0.0:1776189359.804653:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189359.804655:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1776189359.804659:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6775 00000100:00080000:0.0:1776189359.804675:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9501 00000100:00080000:0.0:1776189359.804677:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1776189359.804681:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189359.804686:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008206d902 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664332736:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189359.804690:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189359.804696:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009cace85d x1862459664333056/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189359.804706:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189359.804709:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189359.804710:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189359.804714:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9501 00002000:00000024:3.0:1776189359.804715:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00080000:0.0:1776189359.804716:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1776189359.804722:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189359.804732:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b9092100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664332672:12345-0@lo:13:osp-pre-0-1.0 Request processed in 82us (238us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189359.804738:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000723457d6 x1862459664333120/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189359.804753:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6774 00000100:00100000:1.0:1776189359.804782:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a6e7eea6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664333184:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189359.804802:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000050c951fb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664332864:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189359.804811:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189359.804846:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189359.804852:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b45ebec6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664332672:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189359.804872:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002ced68f2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664333312:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189359.804877:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776189359.804896:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189359.804898:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000073262ad2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664332992:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189359.804916:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1776189359.804925:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000862ba2d5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664332800:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189359.804931:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189359.804965:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189359.804969:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664332864 00000100:00100000:0.0:1776189359.804984:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a78be7d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664333248:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189359.805001:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664333312 00000100:00100000:2.0:1776189359.805060:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a680c874 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664332928:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189359.805150:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664332992 00000100:00100000:1.0:1776189359.805166:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9933 00000100:00100000:1.0:1776189359.805170:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000081d5cbd8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664332864:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189359.807289:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189359.807462:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664332800 00000100:00100000:0.0:1776189359.807511:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1445 00000100:00100000:0.0:1776189359.807517:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000089048673 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664332800:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189359.807566:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000089048673 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664332800:12345-0@lo:400:kworker/u8:0.0 Request processed in 50us (2604us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189359.807574:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1445 00000100:00100000:0.0:1776189359.807626:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000862ba2d5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664332800:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189359.807687:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664333184 00000100:00100000:0.0:1776189359.807712:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664333248 00000100:00100000:0.0:1776189359.807730:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8540 00000100:00100000:0.0:1776189359.807733:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c04acb9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664333184:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189359.807774:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c04acb9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664333184:12345-0@lo:400:kworker/u8:0.0 Request processed in 40us (2964us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189359.807780:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8540 00000100:00100000:0.0:1776189359.807792:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8541 00000100:00100000:0.0:1776189359.807796:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000090d63ac9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664333248:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189359.807821:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000090d63ac9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664333248:12345-0@lo:400:kworker/u8:0.0 Request processed in 25us (541us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189359.807826:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8541 00000100:00100000:0.0:1776189359.807856:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a6e7eea6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664333184:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189359.807881:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9934 00000100:00100000:0.0:1776189359.807885:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000095a4d23a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664333312:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189359.807919:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000095a4d23a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664333312:12345-0@lo:400:kworker/u8:0.0 Request processed in 33us (3024us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189359.807923:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9934 00000100:00100000:0.0:1776189359.807938:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9935 00000100:00100000:0.0:1776189359.807942:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000068da6036 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664332992:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189359.807968:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000068da6036 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664332992:12345-0@lo:400:kworker/u8:0.0 Request processed in 25us (3041us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189359.807973:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9935 00000100:00100000:0.0:1776189359.807998:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000073262ad2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664332992:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189359.808018:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a78be7d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664333248:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189359.808027:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002ced68f2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664333312:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189359.811756:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189359.811793:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cace85d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664333056:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189359.811845:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189359.811867:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000723457d6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664333120:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189359.811871:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008206d902 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664332736:12345-0@lo:13:osp-pre-1-1.0 Request processed in 7041us (7155us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189359.811884:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6775 00000100:00100000:2.0:1776189359.811895:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189359.812013:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bd5316a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664332736:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189359.812229:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664332928 00000100:00100000:0.0:1776189359.812261:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664333056 00000100:00100000:0.0:1776189359.812279:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57353 00000100:00100000:0.0:1776189359.812285:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003258ba66 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664332928:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189359.812300:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189359.812347:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003258ba66 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664332928:12345-0@lo:38:kworker/u8:0.0 Request processed in 63us (596us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189359.812883:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57353 00000100:00100000:0.0:1776189359.812905:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57354 00000100:00100000:0.0:1776189359.812909:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000997c7cab pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664333056:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189359.812919:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189359.812951:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000997c7cab pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664333056:12345-0@lo:38:kworker/u8:0.0 Request processed in 40us (1106us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189359.812957:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57354 00000100:00100000:3.0:1776189359.813526:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664333120 00000100:00100000:3.0:1776189359.813700:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9117 00000100:00100000:3.0:1776189359.813705:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b696663d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664333120:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189359.813716:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189359.813771:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b696663d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664333120:12345-0@lo:38:kworker/u8:0.0 Request processed in 64us (1876us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189359.813777:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9117 00000100:00080000:2.0:1776189359.814107:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189359.814112:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189359.814117:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a680c874 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664332928:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776189359.814129:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189359.814132:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189359.814134:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cace85d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664333056:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776189359.814143:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189359.814146:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189359.814148:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000723457d6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664333120:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189359.818165:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000081d5cbd8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664332864:12345-0@lo:400:kworker/u8:0.0 Request processed in 12983us (13310us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189359.818206:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9933 00000100:00100000:0.0:1776189359.818472:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000050c951fb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664332864:0@lo:400:kworker/u8:0.0 00010000:02020000:2.0:1776189359.829090:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189360.085315:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189360.124368:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189360.340282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189360.596304:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189360.852277:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189361.108283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189361.148681:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189361.364283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189361.622107:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189361.876400:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189362.132244:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189362.177696:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189362.388265:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189362.644699:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189362.900589:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189363.156477:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189363.196493:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189363.417063:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189363.578563:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189363.578610:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189363.578655:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189363.578679:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654952768 00000100:00100000:2.0:1776189363.578742:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654952704 00000100:00100000:0.0:1776189363.578750:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654952832 00000100:00100000:0.0:1776189363.578777:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9936 00000100:00100000:0.0:1776189363.578783:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e6402bd2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654952768:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776189363.578791:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1446 00000100:00100000:2.0:1776189363.578797:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000007949e4c pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654952704:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1776189363.578857:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e6402bd2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654952768:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 73us (301us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189363.578865:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9936 00000100:00100000:0.0:1776189363.578881:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57355 00000100:00100000:0.0:1776189363.578884:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b8bc9a84 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654952832:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:0.0:1776189363.578898:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189363.583838:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b8bc9a84 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654952832:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 4947us (5223us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189363.583854:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57355 00000100:00100000:2.0:1776189363.584499:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000007949e4c pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654952704:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 4960us (5103us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189363.584512:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1446 00010000:02020000:2.0:1776189363.669005:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189363.900496:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776189363.900527:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003da19f87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664333376:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189363.900608:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189363.900688:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664333376 00000100:00100000:3.0:1776189363.900748:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6776 00000100:00100000:3.0:1776189363.900755:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002ced68f2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664333376:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189363.900760:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776189363.900770:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000903833d6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664333440:0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189363.900785:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189363.900802:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189363.900833:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002ced68f2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664333376:12345-0@lo:13:osp-pre-1-0.0 Request processed in 76us (224us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189363.900840:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6776 00000100:00100000:3.0:1776189363.900861:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664333440 00000100:00100000:1.0:1776189363.900899:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003da19f87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664333376:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189363.900904:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6777 00000100:00100000:3.0:1776189363.900912:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000040960a3b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664333440:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189363.900929:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189363.900970:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000040960a3b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664333440:12345-0@lo:13:osp-pre-0-0.0 Request processed in 58us (170us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189363.900975:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6777 00000100:00100000:1.0:1776189363.901015:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000903833d6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664333440:0@lo:13:osp-pre-0-0.0 00010000:02020000:2.0:1776189363.924287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189364.180293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189364.221073:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189364.437343:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189364.692293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189364.924595:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776189364.924620:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000efff0bc3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664333504:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189364.924697:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189364.924796:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664333504 00000100:00100000:1.0:1776189364.924860:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6778 00000100:00100000:1.0:1776189364.924866:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000945ba48a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664333504:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189364.924888:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189364.924931:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000945ba48a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664333504:12345-0@lo:13:osp-pre-0-1.0 Request processed in 64us (235us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189364.924941:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6778 00000100:00080000:2.0:1776189364.925108:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189364.925113:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189364.925116:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189364.925121:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9506 00000100:00080000:2.0:1776189364.925123:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776189364.925127:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189364.925145:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000308dffba x1862459664333696/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189364.925168:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189364.925198:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189364.925199:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189364.925202:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9506 00000100:00080000:2.0:1776189364.925204:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776189364.925207:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189364.925218:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000a2a0606 x1862459664333824/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189364.925226:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189364.925229:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189364.925231:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189364.925234:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9506 00000100:00080000:2.0:1776189364.925235:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776189364.925238:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189364.925248:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cd069f70 x1862459664333888/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189364.925289:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009ac11ef3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664333568:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189364.925321:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189364.925338:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b7713fdb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664334016:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189364.925361:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189364.925393:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000efff0bc3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664333504:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189364.925411:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776189364.925415:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e48fdf91 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664333952:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189364.925434:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189364.925455:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed7f8766 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664333760:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189364.925479:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189364.925491:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046184184 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664334144:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189364.925514:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189364.925531:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:2.0:1776189364.925537:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008809b423 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664333632:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189364.925573:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189364.925605:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e5e4d83b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664334080:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189364.925629:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189364.925670:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664333760 00000100:00100000:3.0:1776189364.925686:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664333632 00000100:00100000:2.0:1776189364.925705:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664334080 00000100:00100000:3.0:1776189364.925721:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9937 00000100:00100000:3.0:1776189364.925728:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001830fc6b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664333760:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189364.925773:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001830fc6b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664333760:12345-0@lo:400:kworker/u8:0.0 Request processed in 45us (294us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189364.925825:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664334016 00000100:00100000:0.0:1776189364.925851:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664333952 00000100:00100000:0.0:1776189364.925869:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664334144 00000100:00100000:0.0:1776189364.925886:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8542 00000100:00100000:0.0:1776189364.925889:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007bf8226c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664334016:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189364.925924:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007bf8226c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664334016:12345-0@lo:400:kworker/u8:0.0 Request processed in 33us (562us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189364.925930:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8542 00000100:00100000:0.0:1776189364.925943:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8543 00000100:00100000:0.0:1776189364.925946:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000727dd03d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664333952:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189364.925975:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000727dd03d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664333952:12345-0@lo:400:kworker/u8:0.0 Request processed in 28us (540us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189364.925980:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8543 00000100:00100000:0.0:1776189364.925992:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8544 00000100:00100000:0.0:1776189364.925995:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000054fe574a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664334144:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189364.926017:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000054fe574a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664334144:12345-0@lo:400:kworker/u8:0.0 Request processed in 22us (505us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189364.926022:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8544 00000100:00100000:1.0:1776189364.926062:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664333568 00000100:00100000:1.0:1776189364.926088:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1447 00000100:00100000:1.0:1776189364.926091:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c1af929a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664333568:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189364.926119:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c1af929a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664333568:12345-0@lo:400:kworker/u8:0.0 Request processed in 27us (799us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189364.926125:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1447 00000100:00100000:2.0:1776189364.926200:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9938 00000100:00100000:2.0:1776189364.926204:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000136e6642 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664333632:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189364.926231:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000136e6642 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664333632:12345-0@lo:400:kworker/u8:0.0 Request processed in 26us (658us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189364.926236:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9938 00000100:00100000:2.0:1776189364.926247:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9939 00000100:00100000:2.0:1776189364.926250:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000270ed845 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664334080:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189364.926268:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000270ed845 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664334080:12345-0@lo:400:kworker/u8:0.0 Request processed in 18us (641us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189364.926273:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9939 00000100:00100000:2.0:1776189364.926302:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e48fdf91 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664333952:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189364.926309:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009ac11ef3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664333568:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189364.926316:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b7713fdb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664334016:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189364.926332:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776189364.926337:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000053ddeb10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664334208:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189364.926357:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189364.926381:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664334208 00000100:00100000:2.0:1776189364.926403:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6779 00000100:00100000:2.0:1776189364.926406:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b7713fdb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664334208:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189364.926421:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189364.926451:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b7713fdb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664334208:12345-0@lo:13:osp-pre-1-1.0 Request processed in 44us (94us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189364.926457:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6779 00000100:00100000:2.0:1776189364.926486:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000053ddeb10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664334208:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189364.926506:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008809b423 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664333632:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189364.926515:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e5e4d83b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664334080:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189364.926523:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed7f8766 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664333760:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189364.926528:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046184184 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664334144:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189364.927121:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9937 00000100:00100000:3.0:1776189364.928517:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000308dffba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664333696:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189364.928589:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189364.928616:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000a2a0606 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664333824:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189364.928638:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189364.928653:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd069f70 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664333888:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189364.928676:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189364.928712:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664333696 00000100:00100000:3.0:1776189364.928745:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664333824 00000100:00100000:3.0:1776189364.928761:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57356 00000100:00100000:3.0:1776189364.928767:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007c812403 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664333696:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189364.928782:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189364.930009:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57357 00000100:00100000:2.0:1776189364.930016:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002c2b2af4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664333824:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189364.930029:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189364.930088:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002c2b2af4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664333824:12345-0@lo:38:kworker/u8:0.0 Request processed in 71us (1449us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189364.930096:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57357 00000100:00100000:2.0:1776189364.930146:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664333888 00000100:00100000:2.0:1776189364.930189:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9118 00000100:00100000:2.0:1776189364.930193:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e8ddaf27 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664333888:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189364.930200:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189364.930227:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e8ddaf27 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664333888:12345-0@lo:38:kworker/u8:0.0 Request processed in 32us (1550us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189364.930232:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9118 00000100:00080000:2.0:1776189364.930282:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189364.930286:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189364.930292:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000a2a0606 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664333824:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776189364.930302:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189364.930305:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189364.930307:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd069f70 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664333888:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189364.930724:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007c812403 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664333696:12345-0@lo:38:kworker/u8:0.0 Request processed in 1955us (2134us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189364.930732:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57356 00000100:00080000:2.0:1776189364.933938:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189364.933948:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189364.933955:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000308dffba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664333696:0@lo:38:kworker/u8:0.0 00010000:02020000:2.0:1776189364.949277:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189365.204338:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189365.244395:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189365.461532:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189365.716610:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189365.972278:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189366.228360:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189366.268369:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189366.484414:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189366.742902:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189366.996332:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189367.252280:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189367.292407:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189367.508282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189367.765100:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189368.020287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189368.276586:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189368.316533:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189368.533324:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189368.700843:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189368.700944:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654952896 00000100:00100000:0.0:1776189368.701001:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1448 00000100:00100000:0.0:1776189368.701007:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e2219b01 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654952896:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189368.701080:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e2219b01 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654952896:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 73us (243us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189368.701088:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1448 00000100:00100000:3.0:1776189368.716896:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189368.717491:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654952960 00000100:00100000:1.0:1776189368.717562:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9940 00000100:00100000:1.0:1776189368.717569:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f3b872c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654952960:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189368.717641:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f3b872c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654952960:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 72us (753us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189368.717649:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9940 00000100:00100000:3.0:1776189368.717821:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189368.717868:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654953024 00000100:00100000:2.0:1776189368.717902:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57358 00000100:00100000:2.0:1776189368.717906:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000006c1656a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654953024:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776189368.717922:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189368.717961:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000006c1656a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654953024:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 54us (142us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189368.717967:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57358 00010000:02020000:2.0:1776189368.788293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189369.020511:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776189369.020533:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000002d6a3fa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664334272:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189369.020599:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189369.020701:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664334272 00000100:00100000:1.0:1776189369.020754:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6780 00000100:00100000:1.0:1776189369.020759:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a78be7d4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664334272:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189369.020787:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189369.020829:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a78be7d4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664334272:12345-0@lo:13:osp-pre-1-0.0 Request processed in 70us (231us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189369.020837:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6780 00000100:00100000:0.0:1776189369.020945:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000002d6a3fa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664334272:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189369.021406:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000308dffba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664334336:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189369.021438:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189369.021518:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664334336 00000100:00100000:0.0:1776189369.021557:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6781 00000100:00100000:0.0:1776189369.021561:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000002d6a3fa pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664334336:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776189369.021578:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189369.021622:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000002d6a3fa pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664334336:12345-0@lo:13:osp-pre-0-0.0 Request processed in 60us (185us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189369.021629:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6781 00000100:00100000:2.0:1776189369.021703:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000308dffba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664334336:0@lo:13:osp-pre-0-0.0 00010000:02020000:2.0:1776189369.044280:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189369.300259:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189369.340383:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189369.557869:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189369.812288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189370.044825:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00080000:0.0:1776189370.044828:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189370.044834:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189370.044837:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189370.044845:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9511 00000100:00100000:3.0:1776189370.044846:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b9f0254e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664334400:0@lo:13:osp-pre-1-1.0 00000100:00080000:0.0:1776189370.044849:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00080000:0.0:1776189370.044861:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189370.044879:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000087888c88 x1862459664334592/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189370.044899:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189370.044901:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189370.044903:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189370.044906:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9511 00000100:00080000:0.0:1776189370.044908:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00080000:0.0:1776189370.044911:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189370.044916:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189370.044925:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bd5316a5 x1862459664334720/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189370.044933:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189370.044936:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189370.044937:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189370.044940:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9511 00000100:00080000:0.0:1776189370.044942:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 6s 00000100:00080000:0.0:1776189370.044946:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189370.044951:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e2382dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664334656:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189370.044959:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000050c951fb x1862459664334784/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189370.044995:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189370.045010:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a6e7eea6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664334464:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189370.045019:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000073262ad2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664335040:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189370.045090:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000723457d6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664334848:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189370.045133:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664334400 00000100:00100000:3.0:1776189370.045140:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189370.045189:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6782 00000100:00100000:1.0:1776189370.045196:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053ddeb10 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664334400:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189370.045205:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776189370.045214:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000efff0bc3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664335104:0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189370.045223:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189370.045250:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189370.045264:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053ddeb10 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664334400:12345-0@lo:13:osp-pre-1-1.0 Request processed in 67us (351us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189370.045272:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6782 00000100:00100000:1.0:1776189370.045288:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664335104 00000100:00100000:3.0:1776189370.045295:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b9f0254e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664334400:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189370.045303:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6783 00000100:00100000:1.0:1776189370.045307:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f326e309 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664335104:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189370.045321:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189370.045348:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f326e309 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664335104:12345-0@lo:13:osp-pre-0-1.0 Request processed in 41us (110us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189370.045353:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6783 00000100:00100000:1.0:1776189370.045373:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664334656 00000100:00100000:1.0:1776189370.045393:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9941 00000100:00100000:1.0:1776189370.045396:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000087513f3f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664334656:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189370.045419:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000087513f3f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664334656:12345-0@lo:400:kworker/u8:1.0 Request processed in 23us (438us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189370.045425:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9941 00000100:00100000:1.0:1776189370.045459:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000efff0bc3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664335104:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189370.045470:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e2382dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664334656:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189370.045502:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001e34c1c7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664334528:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189370.045524:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189370.045535:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a680c874 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664334976:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189370.045552:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189370.045571:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664334528 00000100:00100000:1.0:1776189370.045586:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664334976 00000100:00100000:1.0:1776189370.045599:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9942 00000100:00100000:1.0:1776189370.045602:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000efff0bc3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664334528:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189370.045626:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000efff0bc3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664334528:12345-0@lo:400:kworker/u8:1.0 Request processed in 23us (103us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189370.045631:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9942 00000100:00100000:1.0:1776189370.045640:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9943 00000100:00100000:1.0:1776189370.045643:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008e2382dc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664334976:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189370.045660:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008e2382dc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664334976:12345-0@lo:400:kworker/u8:1.0 Request processed in 17us (108us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189370.045665:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9943 00000100:00100000:1.0:1776189370.045681:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001e34c1c7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664334528:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189370.045688:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a680c874 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664334976:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189370.045725:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664335040 00000100:00100000:1.0:1776189370.045743:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8545 00000100:00100000:1.0:1776189370.045746:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002fbab827 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664335040:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189370.045771:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002fbab827 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664335040:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (633us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189370.045776:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8545 00000100:00100000:1.0:1776189370.045794:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000073262ad2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664335040:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189370.046405:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000087888c88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664334592:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189370.046461:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189370.046491:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bd5316a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664334720:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189370.046513:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189370.046530:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000050c951fb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664334784:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189370.046763:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189370.046808:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664334784 00000100:00100000:3.0:1776189370.046836:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9119 00000100:00100000:3.0:1776189370.046841:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f159c11f pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664334784:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189370.046855:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189370.046890:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f159c11f pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664334784:12345-0@lo:38:kworker/u8:1.0 Request processed in 48us (128us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189370.046896:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9119 00000100:00100000:3.0:1776189370.046916:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664334592 00000100:00100000:3.0:1776189370.046933:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664334720 00000100:00100000:3.0:1776189370.046947:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57359 00000100:00100000:3.0:1776189370.046951:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000062a963a6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664334592:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189370.046958:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189370.046982:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000062a963a6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664334592:12345-0@lo:38:kworker/u8:1.0 Request processed in 30us (523us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189370.046987:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57359 00000100:00100000:1.0:1776189370.047687:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57360 00000100:00100000:1.0:1776189370.047698:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b9f0254e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664334720:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189370.047699:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:02000000:1.0:1776189370.047712:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189370.047728:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cace85d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664334912:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189370.047757:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b9f0254e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664334720:12345-0@lo:38:kworker/u8:1.0 Request processed in 62us (1242us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189370.047768:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189370.047768:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57360 00000100:00100000:0.0:1776189370.047811:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664334912 00000100:00100000:0.0:1776189370.047841:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8546 00000100:00100000:0.0:1776189370.047846:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000862ba2d5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664334912:12345-0@lo:400:kworker/u8:1.0 00000100:00080000:1.0:1776189370.047852:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189370.047857:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189370.047862:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000087888c88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664334592:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776189370.047876:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189370.047879:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189370.047882:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bd5316a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664334720:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189370.047886:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000862ba2d5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664334912:12345-0@lo:400:kworker/u8:1.0 Request processed in 38us (120us total) trans 0 rc 0/0 00000100:00080000:1.0:1776189370.047897:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189370.047901:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189370.047904:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000050c951fb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664334784:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189370.047905:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8546 00000100:00100000:0.0:1776189370.047937:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664334464 00000100:00100000:1.0:1776189370.047958:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cace85d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664334912:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189370.047964:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1449 00000100:00100000:0.0:1776189370.047968:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003e184549 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664334464:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189370.048002:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003e184549 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664334464:12345-0@lo:400:kworker/u8:1.0 Request processed in 33us (308us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189370.048009:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1449 00000100:00100000:1.0:1776189370.048032:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a6e7eea6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664334464:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189370.051343:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189370.051471:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664334848 00000100:00100000:0.0:1776189370.051565:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8547 00000100:00100000:0.0:1776189370.051571:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000308dffba pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664334848:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189370.051637:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000308dffba pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664334848:12345-0@lo:400:kworker/u8:1.0 Request processed in 65us (301us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189370.051644:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8547 00000100:00100000:2.0:1776189370.051716:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000723457d6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664334848:0@lo:400:kworker/u8:1.0 00010000:02020000:2.0:1776189370.068282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189370.324300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189370.364506:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189370.584041:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189370.836486:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189371.092571:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189371.348847:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189371.394329:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189371.605130:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189371.862452:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189372.116352:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189372.372299:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189372.413365:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189372.628285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189372.884999:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189373.140253:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189373.396290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189373.436843:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189373.652499:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189373.817976:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189373.818031:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189373.818423:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654953152 00000100:00100000:0.0:1776189373.818483:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9944 00000100:00100000:0.0:1776189373.818490:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fbd6246c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654953152:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1776189373.818569:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fbd6246c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654953152:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 79us (538us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189373.818577:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9944 00000100:00100000:1.0:1776189373.818931:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654953088 00000100:00100000:1.0:1776189373.818972:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1450 00000100:00100000:1.0:1776189373.818977:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000089ddaa97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654953088:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776189373.819009:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000089ddaa97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654953088:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 31us (1041us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189373.819015:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1450 00000100:00100000:3.0:1776189373.820710:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189373.820786:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654953216 00000100:00100000:2.0:1776189373.820831:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57361 00000100:00100000:2.0:1776189373.820837:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cb607afd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654953216:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:2.0:1776189373.820855:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189373.822093:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cb607afd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654953216:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 1242us (1376us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189373.822106:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57361 00010000:02020000:2.0:1776189373.908291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189374.140462:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1776189374.140484:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd069f70 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664335168:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189374.140556:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189374.140589:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a6e7eea6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664335232:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189374.140609:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189374.140691:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664335168 00000100:00100000:3.0:1776189374.140722:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664335232 00000100:00100000:0.0:1776189374.140744:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6784 00000100:00100000:0.0:1776189374.140797:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000087888c88 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664335168:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776189374.140828:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189374.140874:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000087888c88 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664335168:12345-0@lo:13:osp-pre-0-0.0 Request processed in 124us (319us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189374.140930:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd069f70 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664335168:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189374.141501:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6784 00000100:00100000:0.0:1776189374.141522:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6785 00000100:00100000:0.0:1776189374.141527:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001e34c1c7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664335232:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189374.141618:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189374.141657:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001e34c1c7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664335232:12345-0@lo:13:osp-pre-1-0.0 Request processed in 130us (1047us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189374.141663:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6785 00000100:00100000:2.0:1776189374.141747:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a6e7eea6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664335232:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189374.164287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189374.421211:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189374.460417:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189374.676279:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189374.933521:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189375.164395:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a6e7eea6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664335360:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189375.164461:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189375.164493:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664335360 00000100:00100000:2.0:1776189375.164584:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000044d1311d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664335296:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189375.164616:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189375.164753:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664335296 00000100:00080000:0.0:1776189375.164826:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189375.164832:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189375.164835:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189375.164840:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9517 00000100:00080000:0.0:1776189375.164844:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00080000:0.0:1776189375.164848:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189375.164866:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000025a2ff5a x1862459664335552/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189375.164886:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189375.164889:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189375.164890:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189375.164894:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9517 00000100:00080000:0.0:1776189375.164896:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:0.0:1776189375.164899:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189375.164909:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000476e266d x1862459664335680/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189375.164917:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189375.164919:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189375.164920:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189375.164923:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9517 00000100:00080000:0.0:1776189375.164925:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 11s 00000100:00080000:0.0:1776189375.164927:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189375.164937:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b181f293 x1862459664335744/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189375.164971:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f85662d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664335424:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189375.165004:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189375.165021:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d2d0e609 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664335872:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189375.165043:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189375.165059:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776189375.165064:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024430557 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664335808:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189375.165084:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189375.165108:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664335872 00000100:00100000:0.0:1776189375.165146:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664335808 00000100:00100000:0.0:1776189375.165164:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8548 00000100:00100000:0.0:1776189375.165169:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000797ba66f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664335872:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189375.165233:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000797ba66f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664335872:12345-0@lo:400:kworker/u8:1.0 Request processed in 64us (190us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189375.165240:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8548 00000100:00100000:0.0:1776189375.165254:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8549 00000100:00100000:0.0:1776189375.165258:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d6ea221c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664335808:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189375.165281:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d6ea221c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664335808:12345-0@lo:400:kworker/u8:1.0 Request processed in 23us (198us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189375.165287:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8549 00000100:00100000:0.0:1776189375.165309:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008d1771b5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664335488:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189375.165347:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189375.165365:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000def45ccb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664335936:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189375.165389:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189375.165405:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776189375.165410:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001f4e478a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664335616:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189375.165430:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189375.165456:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664335488 00000100:00100000:0.0:1776189375.165481:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664335936 00000100:00100000:0.0:1776189375.165496:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664335616 00000100:00100000:0.0:1776189375.165512:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9945 00000100:00100000:0.0:1776189375.165515:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000723457d6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664335488:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189375.165547:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000723457d6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664335488:12345-0@lo:400:kworker/u8:1.0 Request processed in 31us (200us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189375.165553:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9945 00000100:00100000:0.0:1776189375.165563:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9946 00000100:00100000:0.0:1776189375.165566:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000020ebaeac pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664335936:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189375.165589:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000020ebaeac pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664335936:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (200us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189375.165595:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9946 00000100:00100000:0.0:1776189375.165605:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9947 00000100:00100000:0.0:1776189375.165608:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c68f859e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664335616:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189375.165633:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c68f859e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664335616:12345-0@lo:400:kworker/u8:1.0 Request processed in 23us (202us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189375.165639:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9947 00000100:00100000:0.0:1776189375.165667:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664335424 00000100:00100000:0.0:1776189375.165692:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1451 00000100:00100000:0.0:1776189375.165696:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006e7c9c41 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664335424:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189375.165722:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006e7c9c41 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664335424:12345-0@lo:400:kworker/u8:1.0 Request processed in 25us (718us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189375.165727:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1451 00000100:00100000:0.0:1776189375.165775:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024430557 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664335808:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189375.165784:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f85662d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664335424:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189375.165792:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d2d0e609 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664335872:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189375.165826:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001f4e478a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664335616:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189375.165834:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008d1771b5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664335488:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189375.165842:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000def45ccb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664335936:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189375.165873:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000025a2ff5a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664335552:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189375.165899:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189375.165913:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000476e266d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664335680:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189375.165932:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189375.165945:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b181f293 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664335744:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189375.165964:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189375.165988:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664335744 00000100:00100000:0.0:1776189375.166006:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9120 00000100:00100000:0.0:1776189375.166010:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001f4e478a pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664335744:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189375.166023:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189375.166053:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001f4e478a pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664335744:12345-0@lo:38:kworker/u8:1.0 Request processed in 42us (88us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189375.166058:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9120 00000100:00100000:0.0:1776189375.166072:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664335552 00000100:00100000:0.0:1776189375.166085:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664335680 00000100:00100000:0.0:1776189375.166095:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57362 00000100:00100000:0.0:1776189375.166098:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000def45ccb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664335552:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189375.166104:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189375.166126:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000def45ccb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664335552:12345-0@lo:38:kworker/u8:1.0 Request processed in 27us (228us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189375.166131:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57362 00000100:00100000:0.0:1776189375.166141:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57363 00000100:00100000:0.0:1776189375.166143:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008d1771b5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664335680:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189375.166149:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189375.166170:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008d1771b5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664335680:12345-0@lo:38:kworker/u8:1.0 Request processed in 25us (237us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189375.166203:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57363 00000100:00080000:0.0:1776189375.166228:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189375.166232:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189375.166235:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000025a2ff5a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664335552:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776189375.166244:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189375.166247:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189375.166249:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000476e266d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664335680:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776189375.166256:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189375.166258:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189375.166260:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b181f293 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664335744:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189375.169435:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6786 00000100:00100000:3.0:1776189375.169447:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000086d6a2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664335360:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776189375.169478:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189375.169570:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000086d6a2c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664335360:12345-0@lo:13:osp-pre-0-1.0 Request processed in 124us (5109us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189375.169577:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6786 00000100:00100000:3.0:1776189375.169601:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6787 00000100:00100000:3.0:1776189375.169605:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000a2a0606 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664335296:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189375.169617:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189375.169666:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000a2a0606 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664335296:12345-0@lo:13:osp-pre-1-1.0 Request processed in 60us (5050us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189375.169672:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6787 00000100:00100000:3.0:1776189375.169725:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000044d1311d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664335296:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189375.169932:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a6e7eea6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664335360:0@lo:13:osp-pre-0-1.0 00010000:02020000:0.0:1776189375.189327:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189375.447297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189375.490541:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189375.700288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189375.956593:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189376.212248:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189376.468311:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189376.508442:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189376.724291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189376.980351:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189377.236347:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189377.492379:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189377.532672:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189377.748312:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189378.004408:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189378.260246:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189378.516291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189378.556405:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189378.773344:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189378.941042:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189378.941096:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189378.941133:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189378.941244:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654953280 00000100:00100000:0.0:1776189378.941315:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1452 00000100:00100000:0.0:1776189378.941322:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d3d96460 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654953280:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1776189378.941395:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d3d96460 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654953280:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 72us (298us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189378.941406:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1452 00000100:00100000:2.0:1776189378.946360:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654953344 00000100:00100000:2.0:1776189378.946453:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654953408 00000100:00100000:2.0:1776189378.946492:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9948 00000100:00100000:2.0:1776189378.946499:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000044d1311d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654953344:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776189378.946667:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000044d1311d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654953344:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 167us (5632us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189378.947350:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57364 00000100:00100000:0.0:1776189378.947357:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000030f2672a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654953408:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:0.0:1776189378.947374:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189378.947411:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000030f2672a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654953408:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 54us (6278us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189378.947417:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57364 00000100:00100000:2.0:1776189378.954929:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9948 00010000:02020000:2.0:1776189379.028579:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189379.261880:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776189379.261902:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000045c5468b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664336064:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189379.261933:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776189379.261946:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046184184 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664336000:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189379.261973:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189379.262021:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189379.262057:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664336064 00000100:00100000:2.0:1776189379.262093:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664336000 00000100:00100000:2.0:1776189379.262110:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6788 00000100:00100000:2.0:1776189379.262116:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ed7f8766 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664336064:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189379.262148:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189379.262211:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6789 00000100:00100000:2.0:1776189379.262220:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ed7f8766 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664336064:12345-0@lo:13:osp-pre-1-0.0 Request processed in 101us (247us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189379.262221:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b181f293 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664336000:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189379.262237:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6788 00002000:00000024:1.0:1776189379.262254:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189379.262296:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b181f293 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664336000:12345-0@lo:13:osp-pre-0-0.0 Request processed in 77us (277us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189379.262299:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000045c5468b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664336064:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189379.262306:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6789 00000100:00100000:0.0:1776189379.262381:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046184184 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664336000:0@lo:13:osp-pre-0-0.0 00010000:02020000:2.0:1776189379.285297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189379.540292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189379.580527:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189379.796310:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189380.052294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:3.0:1776189380.284392:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1776189380.284399:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d43194c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664336256:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189380.284399:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009ea5366f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664336192:0@lo:400:kworker/u8:0.0 00000100:00080000:3.0:1776189380.284400:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189380.284404:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189380.284411:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9522 00000100:00080000:3.0:1776189380.284415:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1776189380.284419:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189380.284438:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000029eeaafe x1862459664336320/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189380.284463:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189380.284466:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189380.284468:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189380.284472:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9522 00000100:00080000:3.0:1776189380.284474:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1776189380.284478:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189380.284487:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ba6916cf x1862459664336448/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189380.284507:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189380.284512:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189380.284513:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189380.284517:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9522 00000100:00080000:3.0:1776189380.284519:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1776189380.284523:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189380.284536:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e39f5436 x1862459664336576/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189380.284548:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000029eeaafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664336320:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189380.284943:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189380.285023:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664336192 00000100:00100000:2.0:1776189380.285031:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000476e266d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664336128:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189380.285072:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1453 00000100:00100000:3.0:1776189380.285077:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000027854250 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664336192:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189380.285268:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189380.285273:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000027854250 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664336192:12345-0@lo:400:kworker/u8:0.0 Request processed in 195us (332us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189380.285281:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1453 00000100:00100000:2.0:1776189380.285288:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ced31f93 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664336640:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189380.285316:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189380.285322:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664336128 00000100:00100000:2.0:1776189380.285339:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007bd06d4b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664336896:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189380.285345:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189380.285352:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6790 00000100:00100000:3.0:1776189380.285358:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e5e4d83b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664336128:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189380.285508:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189380.285620:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189380.285624:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189380.285628:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000040e81793 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664336704:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189380.285650:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba6916cf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664336448:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189380.285667:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009ea5366f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664336192:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189380.285668:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189380.285685:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189380.285693:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000476e266d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664336128:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189380.285696:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1776189380.285706:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035a819c3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664336384:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189380.285716:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e39f5436 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664336576:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189380.285719:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1776189380.285723:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e5e4d83b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664336128:12345-0@lo:13:osp-pre-1-1.0 Request processed in 363us (454us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189380.285727:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000026520930 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664336512:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189380.285730:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6790 00000100:00100000:0.0:1776189380.285739:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189380.285759:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189380.285762:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664336256 00000100:00100000:2.0:1776189380.285764:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189380.285766:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b827fed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664336768:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189380.285791:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000033264082 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664336832:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189380.285800:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189380.285818:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189380.285861:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664336640 00000100:00100000:3.0:1776189380.285871:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664336320 00000100:00100000:1.0:1776189380.285881:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664336448 00000100:00100000:3.0:1776189380.285887:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664336384 00000100:00100000:2.0:1776189380.285895:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664336896 00000100:00100000:1.0:1776189380.285903:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664336768 00000100:00100000:0.0:1776189380.285907:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664336704 00000100:00100000:3.0:1776189380.285910:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9949 00000100:00100000:2.0:1776189380.285912:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664336832 00000100:00100000:3.0:1776189380.285916:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000025a2ff5a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664336256:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189380.285925:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9950 00000100:00100000:1.0:1776189380.285932:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f85662d8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664336384:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189380.285939:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8550 00000100:00100000:2.0:1776189380.285944:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008809b423 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664336640:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189380.285955:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8551 00000100:00100000:3.0:1776189380.285962:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000025a2ff5a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664336256:12345-0@lo:400:kworker/u8:0.0 Request processed in 45us (620us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189380.285964:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009ac11ef3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664336896:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189380.285976:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9949 00000100:00100000:1.0:1776189380.285981:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f85662d8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664336384:12345-0@lo:400:kworker/u8:0.0 Request processed in 48us (244us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189380.285989:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008809b423 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664336640:12345-0@lo:400:kworker/u8:0.0 Request processed in 43us (672us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189380.285992:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9950 00000100:00100000:2.0:1776189380.286001:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8550 00000100:00100000:3.0:1776189380.286001:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9951 00000100:00100000:3.0:1776189380.286006:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000024430557 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664336768:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189380.286013:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57365 00000100:00100000:0.0:1776189380.286018:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009ac11ef3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664336896:12345-0@lo:400:kworker/u8:0.0 Request processed in 55us (396us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189380.286019:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006080a246 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664336320:12345-0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189380.286024:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8552 00000100:00100000:2.0:1776189380.286031:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b9ada007 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664336832:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189380.286035:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8551 00010000:02000000:1.0:1776189380.286036:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189380.286056:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8553 00000100:00100000:0.0:1776189380.286060:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d2d0e609 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664336704:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189380.286999:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000024430557 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664336768:12345-0@lo:400:kworker/u8:0.0 Request processed in 990us (1199us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189380.287009:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9951 00000100:00100000:1.0:1776189380.287030:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006080a246 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664336320:12345-0@lo:38:kworker/u8:0.0 Request processed in 1011us (1412us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189380.287036:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57366 00000100:00100000:3.0:1776189380.287041:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000558e4f37 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664336448:12345-0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189380.287046:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57365 00000100:00080000:1.0:1776189380.287078:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189380.287080:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189380.287083:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000029eeaafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664336320:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189380.287122:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664336576 00000100:00100000:0.0:1776189380.287138:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d2d0e609 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664336704:12345-0@lo:400:kworker/u8:0.0 Request processed in 1074us (1472us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189380.287142:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9121 00000100:00100000:1.0:1776189380.287147:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000694c72a3 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664336576:12345-0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189380.287151:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8553 00010000:02000000:1.0:1776189380.287155:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189380.287196:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000694c72a3 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664336576:12345-0@lo:38:kworker/u8:0.0 Request processed in 49us (1442us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189380.287201:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9121 00000100:00080000:1.0:1776189380.287242:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189380.287246:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189380.287250:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e39f5436 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664336576:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189380.287258:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035a819c3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664336384:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189380.287273:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b827fed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664336768:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189380.287285:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000040e81793 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664336704:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189380.287296:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d43194c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664336256:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189380.287306:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664336512 00000100:00100000:1.0:1776189380.287334:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6791 00000100:00100000:1.0:1776189380.287339:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e48fdf91 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3364:x1862459664336512:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189380.287355:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189380.287382:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b9ada007 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+6:3364:x1862459664336832:12345-0@lo:400:kworker/u8:0.0 Request processed in 1349us (1564us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189380.287390:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8552 00000100:00100000:0.0:1776189380.287392:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ced31f93 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664336640:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189380.287399:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e48fdf91 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3364:x1862459664336512:12345-0@lo:13:osp-pre-0-1.0 Request processed in 58us (1635us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189380.287407:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6791 00000100:00100000:0.0:1776189380.287413:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007bd06d4b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664336896:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189380.287444:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000026520930 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664336512:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189380.287453:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000033264082 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664336832:0@lo:400:kworker/u8:0.0 00010000:02000000:3.0:1776189380.287486:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189380.287515:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000558e4f37 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664336448:12345-0@lo:38:kworker/u8:0.0 Request processed in 472us (1832us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189380.287521:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57366 00000100:00080000:1.0:1776189380.287589:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189380.287593:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189380.287596:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba6916cf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664336448:0@lo:38:kworker/u8:0.0 00010000:02020000:2.0:1776189380.308513:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189380.564297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189380.604535:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189380.824838:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189381.076301:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189381.332299:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189381.588857:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189381.628463:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189381.847617:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189382.100323:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189382.356391:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189382.612280:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189382.653029:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189382.869831:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189383.124826:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189383.380392:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189383.636336:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189383.676730:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189383.892271:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189384.058088:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189384.058149:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189384.058215:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654953536 00000100:00100000:0.0:1776189384.058266:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9952 00000100:00100000:0.0:1776189384.058272:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000027d4ac99 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654953536:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189384.058288:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189384.058407:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000027d4ac99 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654953536:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 134us (327us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189384.058416:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9952 00000100:00100000:0.0:1776189384.058434:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654953600 00000100:00100000:0.0:1776189384.058460:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57367 00000100:00100000:0.0:1776189384.058464:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008e312a63 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654953600:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776189384.058477:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189384.058504:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008e312a63 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654953600:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 39us (217us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189384.058509:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57367 00000100:00100000:1.0:1776189384.060950:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654953472 00000100:00100000:1.0:1776189384.061020:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1454 00000100:00100000:1.0:1776189384.061026:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000192d1549 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654953472:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189384.061087:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000192d1549 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654953472:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 60us (2948us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189384.061094:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1454 00010000:02020000:0.0:1776189384.149439:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189384.384282:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004088ae05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664336960:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189384.384364:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189384.384499:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664336960 00000100:00100000:2.0:1776189384.384550:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6792 00000100:00100000:2.0:1776189384.384555:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000037fc1275 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664336960:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189384.384585:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189384.384627:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000037fc1275 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664336960:12345-0@lo:13:osp-pre-0-0.0 Request processed in 71us (264us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189384.384633:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6792 00000100:00100000:0.0:1776189384.385421:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004088ae05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664336960:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189384.385442:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776189384.385448:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a3202503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664337024:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189384.385476:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189384.385553:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664337024 00000100:00100000:2.0:1776189384.385589:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6793 00000100:00100000:2.0:1776189384.385593:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005ecd069c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664337024:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189384.385611:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189384.385651:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005ecd069c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664337024:12345-0@lo:13:osp-pre-1-0.0 Request processed in 57us (175us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189384.385657:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6793 00000100:00100000:0.0:1776189384.385852:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a3202503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664337024:0@lo:13:osp-pre-1-0.0 00010000:02020000:0.0:1776189384.404286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189384.660283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189384.703433:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189384.916356:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189385.172279:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189385.404495:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a7066b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664337088:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189385.404575:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189385.404635:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002477597b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664337152:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189385.404671:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189385.404720:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664337152 00000100:00100000:1.0:1776189385.404775:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1455 00000100:00100000:1.0:1776189385.404781:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b64e4d66 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664337152:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189385.404844:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b64e4d66 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664337152:12345-0@lo:400:kworker/u8:0.0 Request processed in 63us (172us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189385.404852:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1455 00000100:00100000:1.0:1776189385.404898:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002477597b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664337152:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189385.404932:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664337088 00000100:00100000:1.0:1776189385.404961:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6794 00000100:00100000:1.0:1776189385.404965:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ee307f96 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664337088:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189385.404981:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189385.407309:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012b5e917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664337216:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189385.407366:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189385.407416:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664337216 00000100:00100000:0.0:1776189385.407455:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6795 00000100:00100000:0.0:1776189385.407461:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cab21754 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664337216:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189385.407479:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189385.407520:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cab21754 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664337216:12345-0@lo:13:osp-pre-1-1.0 Request processed in 58us (154us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189385.407529:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6795 00000100:00100000:0.0:1776189385.407982:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012b5e917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664337216:0@lo:13:osp-pre-1-1.0 00000100:00080000:2.0:1776189385.408184:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189385.408190:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189385.408193:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189385.408198:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9527 00000100:00080000:2.0:1776189385.408201:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1776189385.408245:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c995aa10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664337280:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189385.408279:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189385.408312:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664337280 00000100:00100000:0.0:1776189385.408340:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9953 00000100:00100000:0.0:1776189385.408344:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000012b5e917 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664337280:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189385.408393:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000012b5e917 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664337280:12345-0@lo:400:kworker/u8:0.0 Request processed in 47us (113us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189385.408395:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ee307f96 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664337088:12345-0@lo:13:osp-pre-0-1.0 Request processed in 3426us (3819us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189385.408395:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a7066b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664337088:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189385.408408:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6794 00000100:00100000:0.0:1776189385.408411:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9953 00000100:00100000:0.0:1776189385.408447:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c995aa10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664337280:0@lo:400:kworker/u8:0.0 00000100:00080000:2.0:1776189385.408596:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189385.408617:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c995aa10 x1862459664337344/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189385.408668:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189385.408673:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189385.408674:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189385.408677:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9527 00000100:00080000:2.0:1776189385.408679:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776189385.408683:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189385.408694:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004088ae05 x1862459664337472/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189385.408702:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189385.408705:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189385.408706:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189385.408709:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9527 00000100:00080000:2.0:1776189385.408710:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776189385.408713:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189385.408722:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000046184184 x1862459664337536/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189385.408767:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000997c7cab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664337664:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189385.408796:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189385.408813:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776189385.408817:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000136e6642 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664337728:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189385.408837:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189385.408859:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664337664 00000100:00100000:2.0:1776189385.408884:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8554 00000100:00100000:2.0:1776189385.408888:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000270ed845 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664337664:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189385.408923:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000270ed845 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664337664:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (127us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189385.408928:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8554 00000100:00100000:2.0:1776189385.408951:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000997c7cab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664337664:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189385.409066:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd069f70 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664337600:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189385.409091:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189385.409120:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776189385.409124:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002477597b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664337408:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189385.409145:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189385.409167:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664337728 00000100:00100000:0.0:1776189385.409210:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664337408 00000100:00100000:0.0:1776189385.409226:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9954 00000100:00100000:0.0:1776189385.409230:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001830fc6b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664337728:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189385.409260:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001830fc6b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664337728:12345-0@lo:400:kworker/u8:0.0 Request processed in 30us (424us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189385.409267:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9954 00000100:00100000:0.0:1776189385.409278:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9955 00000100:00100000:0.0:1776189385.409281:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fb6b7b14 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664337408:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189385.409308:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fb6b7b14 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664337408:12345-0@lo:400:kworker/u8:0.0 Request processed in 26us (163us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189385.409314:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9955 00000100:00100000:0.0:1776189385.409344:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002477597b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664337408:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189385.409419:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c995aa10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664337344:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189385.409457:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189385.409476:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004088ae05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664337472:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189385.409496:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189385.409509:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046184184 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664337536:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189385.409530:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189385.409646:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664337536 00000100:00100000:1.0:1776189385.409666:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9122 00000100:00100000:1.0:1776189385.409670:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d4966c1b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664337536:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189385.409683:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189385.409727:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664337344 00000100:00100000:3.0:1776189385.409748:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664337472 00000100:00100000:3.0:1776189385.409761:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57368 00000100:00100000:3.0:1776189385.409764:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000047dd36ff pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664337344:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189385.409770:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189385.409798:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000047dd36ff pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664337344:12345-0@lo:38:kworker/u8:0.0 Request processed in 33us (341us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189385.409803:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57368 00000100:00100000:3.0:1776189385.409813:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57369 00000100:00100000:3.0:1776189385.409815:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c9c38316 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664337472:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189385.409820:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189385.409835:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c9c38316 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664337472:12345-0@lo:38:kworker/u8:0.0 Request processed in 19us (340us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189385.409840:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57369 00000100:00080000:3.0:1776189385.409861:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189385.409863:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189385.409866:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c995aa10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664337344:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776189385.409872:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189385.409874:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189385.409876:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004088ae05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664337472:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189385.410080:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d4966c1b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664337536:12345-0@lo:38:kworker/u8:0.0 Request processed in 408us (549us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189385.410086:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664337600 00000100:00100000:1.0:1776189385.410087:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9122 00000100:00080000:3.0:1776189385.410155:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189385.410159:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189385.410162:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046184184 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664337536:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189385.410213:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000136e6642 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664337728:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189385.410314:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8555 00000100:00100000:2.0:1776189385.410319:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000028933edf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664337600:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189385.410355:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000028933edf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664337600:12345-0@lo:400:kworker/u8:0.0 Request processed in 35us (1263us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189385.410420:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd069f70 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664337600:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189385.410823:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8555 00010000:02020000:0.0:1776189385.428911:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189385.684304:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189385.724576:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189385.940854:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189386.196295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189386.452251:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189386.708286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189386.748483:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189386.967232:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189387.220363:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189387.476286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189387.733590:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189387.774153:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189387.988287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189388.245488:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189388.500336:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189388.757189:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189388.796384:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189389.012577:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189389.183241:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189389.183541:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654953664 00000100:00100000:1.0:1776189389.183601:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1456 00000100:00100000:1.0:1776189389.183607:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f37c27d2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654953664:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776189389.183827:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f37c27d2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654953664:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 218us (594us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189389.183836:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1456 00000100:00100000:3.0:1776189389.186534:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189389.186928:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654953728 00000100:00100000:1.0:1776189389.186986:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9956 00000100:00100000:1.0:1776189389.186993:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0fb2966 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654953728:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776189389.187079:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0fb2966 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654953728:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 86us (553us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189389.187087:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9956 00000100:00100000:3.0:1776189389.194009:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189389.196781:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654953792 00000100:00100000:0.0:1776189389.196853:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57370 00000100:00100000:0.0:1776189389.196860:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000911b4242 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654953792:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:0.0:1776189389.196880:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189389.196956:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000911b4242 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654953792:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 95us (2955us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189389.196963:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57370 00010000:02020000:0.0:1776189389.268431:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189389.503738:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000997c7cab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664337792:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189389.503822:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189389.503869:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776189389.503875:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aac1837c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664337856:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189389.503900:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189389.503952:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664337792 00000100:00100000:2.0:1776189389.503997:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664337856 00000100:00100000:2.0:1776189389.504150:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6796 00000100:00100000:2.0:1776189389.504156:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba6916cf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664337792:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189389.504210:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189389.504254:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba6916cf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664337792:12345-0@lo:13:osp-pre-0-0.0 Request processed in 97us (433us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189389.504261:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6796 00000100:00100000:2.0:1776189389.504277:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6797 00000100:00100000:2.0:1776189389.504281:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000033264082 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664337856:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189389.504295:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189389.504316:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000033264082 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664337856:12345-0@lo:13:osp-pre-1-0.0 Request processed in 35us (417us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189389.504321:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6797 00000100:00100000:3.0:1776189389.504406:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aac1837c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664337856:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189389.504419:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000997c7cab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664337792:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189389.526724:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189389.780384:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189389.820726:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189390.043308:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189390.300338:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189390.524368:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cace85d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664337920:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189390.524434:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189390.524589:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776189390.524596:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000631ae276 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664337984:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189390.524620:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776189390.525468:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189390.525473:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189390.525476:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189390.525481:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9532 00000100:00080000:1.0:1776189390.525484:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189390.525558:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189390.525580:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003f67f1c5 x1862459664338176/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189390.525604:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189390.525607:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189390.525609:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189390.525612:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9532 00000100:00080000:1.0:1776189390.525614:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189390.525617:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189390.525626:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000095a4d23a x1862459664338304/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189390.525633:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189390.525636:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189390.525637:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189390.525640:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9532 00000100:00080000:1.0:1776189390.525642:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189390.525644:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189390.525654:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000081d5cbd8 x1862459664338368/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189390.525698:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004f903da2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664338048:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189390.525735:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189390.525753:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aac1837c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664338496:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189390.525773:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189390.525792:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1776189390.525797:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c2578d77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664338112:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189390.525819:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189390.525833:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a7066b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664338560:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189390.525851:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189390.525881:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664338048 00000100:00100000:1.0:1776189390.525916:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1457 00000100:00100000:1.0:1776189390.525921:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002e45a4e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664338048:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189390.525959:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002e45a4e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664338048:12345-0@lo:400:kworker/u8:1.0 Request processed in 37us (223us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189390.525965:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1457 00000100:00100000:1.0:1776189390.525993:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664338496 00000100:00100000:1.0:1776189390.526027:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8556 00000100:00100000:1.0:1776189390.526031:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000026520930 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664338496:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189390.526058:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000026520930 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664338496:12345-0@lo:400:kworker/u8:1.0 Request processed in 27us (284us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189390.526066:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8556 00000100:00100000:1.0:1776189390.526084:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664338112 00000100:00100000:1.0:1776189390.526106:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664338560 00000100:00100000:1.0:1776189390.526123:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9957 00000100:00100000:1.0:1776189390.526126:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000045c5468b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664338112:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189390.526154:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000045c5468b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664338112:12345-0@lo:400:kworker/u8:1.0 Request processed in 26us (334us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189390.526159:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9957 00000100:00100000:1.0:1776189390.526168:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9958 00000100:00100000:1.0:1776189390.526200:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a6e7eea6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664338560:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189390.526227:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a6e7eea6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664338560:12345-0@lo:400:kworker/u8:1.0 Request processed in 55us (375us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189390.526232:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9958 00000100:00100000:1.0:1776189390.526274:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c2578d77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664338112:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189390.526283:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a7066b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664338560:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189390.526290:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004f903da2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664338048:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189390.526297:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aac1837c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664338496:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189390.526348:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003f67f1c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664338176:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189390.526369:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189390.526382:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000095a4d23a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664338304:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189390.526399:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189390.526410:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000081d5cbd8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664338368:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189390.526432:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189390.526455:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664338368 00000100:00100000:1.0:1776189390.526474:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9123 00000100:00100000:1.0:1776189390.526477:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008a7066b0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664338368:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189390.527485:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f3cbce47 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664338240:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189390.527521:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189390.527538:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a3202503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664338624:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189390.527561:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189390.527577:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776189390.527582:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e2382dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664338432:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189390.527602:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189390.527635:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664338176 00000100:00100000:3.0:1776189390.527657:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664338304 00000100:00100000:3.0:1776189390.527670:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664338240 00000100:00100000:3.0:1776189390.527690:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9959 00000100:00100000:3.0:1776189390.527695:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005d43194c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664338240:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189390.527734:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005d43194c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664338240:12345-0@lo:400:kworker/u8:1.0 Request processed in 38us (213us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189390.527740:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9959 00000100:00100000:3.0:1776189390.527754:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57371 00000100:00100000:3.0:1776189390.527757:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000aac1837c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664338176:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189390.527768:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189390.527800:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000aac1837c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664338176:12345-0@lo:38:kworker/u8:1.0 Request processed in 42us (1431us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189390.527806:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57371 00000100:00100000:3.0:1776189390.527818:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57372 00000100:00100000:3.0:1776189390.527834:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004f903da2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664338304:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189390.527841:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189390.527864:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004f903da2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664338304:12345-0@lo:38:kworker/u8:1.0 Request processed in 29us (1465us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189390.527869:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57372 00000100:00100000:3.0:1776189390.527893:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664338624 00000100:00100000:3.0:1776189390.528071:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664338432 00000100:00100000:3.0:1776189390.528089:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8557 00000100:00100000:3.0:1776189390.528093:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000040e81793 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664338624:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189390.528125:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000040e81793 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664338624:12345-0@lo:400:kworker/u8:1.0 Request processed in 30us (563us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189390.528134:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8557 00000100:00100000:3.0:1776189390.528145:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8558 00000100:00100000:3.0:1776189390.528149:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009b827fed pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664338432:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189390.528275:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009b827fed pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664338432:12345-0@lo:400:kworker/u8:1.0 Request processed in 124us (672us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189390.528282:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8558 00000100:00080000:3.0:1776189390.528320:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189390.528324:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189390.528327:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003f67f1c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664338176:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776189390.528335:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189390.528338:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189390.528341:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000095a4d23a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664338304:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189390.528371:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e2382dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664338432:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189390.528379:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f3cbce47 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664338240:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189390.528387:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a3202503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664338624:0@lo:400:kworker/u8:1.0 00010000:02000000:1.0:1776189390.528627:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189390.528662:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008a7066b0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664338368:12345-0@lo:38:kworker/u8:1.0 Request processed in 2183us (2229us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189390.528669:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9123 00000100:00080000:1.0:1776189390.528693:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189390.528697:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189390.528700:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000081d5cbd8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664338368:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189390.531011:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664337920 00000100:00100000:0.0:1776189390.531083:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664337984 00000100:00100000:0.0:1776189390.531101:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6798 00000100:00100000:0.0:1776189390.531106:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007bd06d4b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664337920:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189390.531130:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189390.531150:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6799 00000100:00100000:1.0:1776189390.531157:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ced31f93 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664337984:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189390.531187:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189390.531244:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ced31f93 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664337984:12345-0@lo:13:osp-pre-0-1.0 Request processed in 87us (6624us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189390.531252:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6799 00000100:00100000:1.0:1776189390.531315:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000631ae276 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664337984:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189390.531887:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007bd06d4b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664337920:12345-0@lo:13:osp-pre-1-1.0 Request processed in 779us (7453us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189390.532132:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cace85d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664337920:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189390.534395:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6798 00010000:02020000:2.0:1776189390.558451:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189390.812286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189390.845379:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189391.068265:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189391.325659:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189391.580644:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189391.836325:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189391.869318:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189392.092287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189392.348287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189392.604288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189392.860281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189392.896340:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189393.116299:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189393.372282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189393.628797:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189393.884288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189393.917889:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189394.140929:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189394.300276:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189394.300454:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654953920 00000100:00100000:3.0:1776189394.300511:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9960 00000100:00100000:3.0:1776189394.300518:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000050c951fb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654953920:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189394.300595:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000050c951fb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654953920:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 77us (327us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189394.300602:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9960 00000100:00100000:3.0:1776189394.311839:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189394.311884:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189394.311975:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654953856 00000100:00100000:1.0:1776189394.311989:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654953984 00000100:00100000:1.0:1776189394.312032:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57373 00000100:00100000:2.0:1776189394.312038:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1458 00000100:00100000:1.0:1776189394.312042:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000035a819c3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654953984:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00000100:00100000:2.0:1776189394.312049:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002ea0417b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654953856:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00010000:02000000:1.0:1776189394.312071:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189394.312116:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002ea0417b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654953856:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 69us (284us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189394.312124:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1458 00000100:00100000:1.0:1776189394.312130:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000035a819c3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654953984:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 85us (242us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189394.312138:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57373 00010000:02020000:2.0:1776189394.396987:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189394.620469:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000903833d6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664338688:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189394.620549:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189394.620834:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664338688 00000100:00100000:2.0:1776189394.620886:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6800 00000100:00100000:2.0:1776189394.620892:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000073262ad2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664338688:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189394.620930:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189394.620971:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000073262ad2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664338688:12345-0@lo:13:osp-pre-0-0.0 Request processed in 78us (422us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189394.620978:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6800 00000100:00100000:3.0:1776189394.621240:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776189394.621251:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000631ae276 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664338752:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189394.621279:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189394.621385:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664338752 00000100:00100000:0.0:1776189394.621419:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6801 00000100:00100000:0.0:1776189394.621423:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e39f5436 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664338752:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189394.621439:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189394.621477:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e39f5436 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664338752:12345-0@lo:13:osp-pre-1-0.0 Request processed in 53us (198us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189394.621483:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6801 00000100:00100000:0.0:1776189394.621531:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000903833d6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664338688:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189394.621663:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000631ae276 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664338752:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189394.653280:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189394.909645:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189394.945954:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189395.164271:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189395.420283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:3.0:1776189395.644467:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189395.644475:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189395.644478:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189395.644486:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9537 00000100:00080000:3.0:1776189395.644490:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776189395.644495:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189395.644510:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776189395.644517:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d4da5ec0 x1862459664339008/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189395.644532:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000029eeaafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664338880:0@lo:400:kworker/u8:0.0 00000100:00080000:3.0:1776189395.644542:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189395.644545:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189395.644547:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189395.644551:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9537 00000100:00080000:3.0:1776189395.644552:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776189395.644555:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189395.644568:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000264516d x1862459664339136/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189395.644575:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189395.644578:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189395.644580:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189395.644583:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9537 00000100:00080000:3.0:1776189395.644585:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776189395.644588:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189395.644600:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000031aca57e x1862459664339200/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189395.644616:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189395.644647:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa8befb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664339072:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189395.644652:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009ea5366f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664338944:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189395.644683:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189395.644712:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000060a8344d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664339456:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189395.644713:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664339072 00000100:00100000:2.0:1776189395.644742:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189395.644757:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9961 00000100:00100000:2.0:1776189395.644760:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1776189395.644765:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000903833d6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664339072:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189395.644768:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a50d0fa6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664339264:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189395.644791:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189395.644806:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002460f3d1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664339520:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189395.644826:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189395.644860:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664339456 00000100:00100000:1.0:1776189395.644961:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d4da5ec0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664339008:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189395.644989:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189395.645004:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000264516d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664339136:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189395.645020:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189395.645031:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000031aca57e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664339200:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189395.645048:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189395.645071:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664339264 00000100:00100000:1.0:1776189395.645094:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664339520 00000100:00100000:1.0:1776189395.645107:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8559 00000100:00100000:1.0:1776189395.645111:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cd069f70 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664339264:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189395.645156:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cd069f70 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664339264:12345-0@lo:400:kworker/u8:0.0 Request processed in 45us (365us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189395.645163:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8559 00000100:00100000:3.0:1776189395.645486:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000903833d6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664339072:12345-0@lo:400:kworker/u8:0.0 Request processed in 722us (805us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189395.645493:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9961 00000100:00100000:3.0:1776189395.645505:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664339008 00000100:00100000:3.0:1776189395.645519:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664339136 00000100:00100000:3.0:1776189395.645532:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57374 00000100:00100000:3.0:1776189395.645535:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000081d5cbd8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664339008:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189395.645547:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189395.645577:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000081d5cbd8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664339008:12345-0@lo:38:kworker/u8:0.0 Request processed in 41us (587us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189395.645583:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57374 00000100:00100000:3.0:1776189395.645594:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57375 00000100:00100000:3.0:1776189395.645597:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c2578d77 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664339136:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189395.645604:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189395.645621:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c2578d77 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664339136:12345-0@lo:38:kworker/u8:0.0 Request processed in 23us (601us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189395.645626:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57375 00000100:00100000:2.0:1776189395.648616:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8560 00000100:00100000:2.0:1776189395.648626:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000136e6642 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664339520:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189395.648709:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000136e6642 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664339520:12345-0@lo:400:kworker/u8:0.0 Request processed in 84us (3882us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189395.648716:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8560 00000100:00100000:2.0:1776189395.648734:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8561 00000100:00100000:2.0:1776189395.648738:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000997c7cab pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664339456:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189395.648765:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000997c7cab pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664339456:12345-0@lo:400:kworker/u8:0.0 Request processed in 27us (4024us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189395.648771:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8561 00000100:00100000:2.0:1776189395.648798:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189395.648828:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005bd91e44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664339328:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189395.648858:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189395.649106:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a680c874 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664339584:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189395.649134:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189395.649154:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1776189395.649161:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003da19f87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664338816:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189395.649421:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189395.649439:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000db3523c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664339392:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189395.649465:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189395.649490:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664339584 00000100:00100000:2.0:1776189395.649518:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664338816 00000100:00100000:2.0:1776189395.649534:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6802 00000100:00100000:2.0:1776189395.649537:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c995aa10 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664339584:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189395.649564:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189395.649816:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c995aa10 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664339584:12345-0@lo:13:osp-pre-1-1.0 Request processed in 277us (682us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189395.649828:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6802 00000100:00100000:2.0:1776189395.649843:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6803 00000100:00100000:2.0:1776189395.649846:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002477597b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664338816:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189395.649858:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189395.650303:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002477597b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664338816:12345-0@lo:13:osp-pre-0-1.0 Request processed in 454us (882us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189395.650311:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6803 00000100:00100000:2.0:1776189395.650341:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664339328 00000100:00100000:2.0:1776189395.650457:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8562 00000100:00100000:2.0:1776189395.650461:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004088ae05 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664339328:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189395.650494:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004088ae05 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664339328:12345-0@lo:400:kworker/u8:0.0 Request processed in 32us (1635us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189395.650500:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8562 00000100:00100000:2.0:1776189395.650521:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664338944 00000100:00100000:2.0:1776189395.650539:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664339392 00000100:00100000:2.0:1776189395.650549:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9962 00000100:00100000:2.0:1776189395.650552:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000046184184 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664338944:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189395.650808:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000046184184 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664338944:12345-0@lo:400:kworker/u8:0.0 Request processed in 253us (2009us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189395.650815:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9962 00000100:00100000:2.0:1776189395.650829:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9963 00000100:00100000:2.0:1776189395.650832:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001be0620a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664339392:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189395.650859:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001be0620a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664339392:12345-0@lo:400:kworker/u8:0.0 Request processed in 27us (1394us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189395.650865:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9963 00000100:00100000:2.0:1776189395.650932:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003da19f87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664338816:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189395.650943:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000db3523c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664339392:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189395.650957:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a680c874 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664339584:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189395.651050:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009ea5366f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664338944:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189395.651060:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005bd91e44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664339328:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189395.651170:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664339200 00000100:00100000:2.0:1776189395.651423:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9124 00000100:00100000:2.0:1776189395.651427:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000631ae276 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664339200:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189395.651439:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189395.651471:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000631ae276 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664339200:12345-0@lo:38:kworker/u8:0.0 Request processed in 44us (6423us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189395.651476:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9124 00000100:00100000:2.0:1776189395.651510:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a50d0fa6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664339264:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189395.651520:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002460f3d1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664339520:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189395.651529:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa8befb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664339072:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189395.651537:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000060a8344d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664339456:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189395.651603:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664338880 00000100:00100000:2.0:1776189395.651628:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1459 00000100:00100000:2.0:1776189395.651632:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a9501f9f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664338880:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189395.651664:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a9501f9f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664338880:12345-0@lo:400:kworker/u8:0.0 Request processed in 31us (7052us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189395.651669:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1459 00000100:00100000:2.0:1776189395.651706:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000029eeaafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664338880:0@lo:400:kworker/u8:0.0 00000100:00080000:2.0:1776189395.651740:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189395.651744:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189395.651747:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d4da5ec0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664339008:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776189395.651758:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189395.651760:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189395.651762:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000264516d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664339136:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776189395.651771:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189395.651773:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189395.651776:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000031aca57e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664339200:0@lo:38:kworker/u8:0.0 00010000:02020000:2.0:1776189395.676309:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189395.932406:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189395.964446:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189396.188281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189396.445483:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189396.702893:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189396.958219:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189396.988509:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189397.212294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189397.469916:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189397.724868:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189397.980858:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189398.013268:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189398.238046:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189398.492275:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189398.748782:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189399.004398:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189399.046430:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189399.260289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189399.419044:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189399.419076:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189399.419110:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189399.419203:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654954176 00000100:00100000:2.0:1776189399.419241:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654954112 00000100:00100000:2.0:1776189399.419273:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9964 00000100:00100000:2.0:1776189399.419279:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009cace85d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654954112:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189399.419342:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009cace85d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654954112:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 63us (266us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189399.419350:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9964 00000100:00100000:2.0:1776189399.419366:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57376 00000100:00100000:2.0:1776189399.419370:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a680c874 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654954176:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776189399.419383:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189399.419406:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a680c874 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654954176:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 35us (369us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189399.419412:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57376 00000100:00100000:1.0:1776189399.419697:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654954048 00000100:00100000:1.0:1776189399.419727:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1460 00000100:00100000:1.0:1776189399.419731:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004c8f754f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654954048:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189399.419763:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004c8f754f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654954048:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 31us (652us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189399.419768:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1460 00010000:02020000:2.0:1776189399.517403:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189399.740392:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a3202503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664339648:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189399.740828:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189399.740941:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664339648 00000100:00100000:1.0:1776189399.741024:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6804 00000100:00100000:1.0:1776189399.741031:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006235ac3c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664339648:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189399.741060:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189399.741105:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006235ac3c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664339648:12345-0@lo:13:osp-pre-1-0.0 Request processed in 74us (278us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189399.741113:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6804 00000100:00100000:0.0:1776189399.741233:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a3202503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664339648:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189399.741258:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776189399.741269:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fb1c1d52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664339712:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189399.741296:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189399.741373:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664339712 00000100:00100000:0.0:1776189399.741405:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6805 00000100:00100000:0.0:1776189399.741410:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000011679d4d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664339712:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776189399.741427:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189399.741684:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000011679d4d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664339712:12345-0@lo:13:osp-pre-0-0.0 Request processed in 273us (388us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189399.741691:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6805 00000100:00100000:2.0:1776189399.741753:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fb1c1d52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664339712:0@lo:13:osp-pre-0-0.0 00010000:02020000:2.0:1776189399.772284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189400.028881:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189400.060456:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189400.284316:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189400.542659:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:3.0:1776189400.764428:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189400.764436:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189400.764439:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189400.764447:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9542 00000100:00080000:3.0:1776189400.764451:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00080000:3.0:1776189400.764457:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189400.764476:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e6665a35 x1862459664339904/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189400.764497:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189400.764500:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189400.764502:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189400.764505:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9542 00000100:00080000:3.0:1776189400.764507:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00080000:3.0:1776189400.764510:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189400.764519:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000822c622a x1862459664340032/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189400.764526:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189400.764529:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189400.764531:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189400.764533:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9542 00000100:00080000:3.0:1776189400.764535:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 6s 00000100:00080000:3.0:1776189400.764538:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189400.764548:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c76be6ad x1862459664340096/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189400.764601:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008cc6ee78 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664339968:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189400.764660:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189400.764687:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003b5d789a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664340352:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189400.764711:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189400.764733:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1776189400.764738:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002e4c5085 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664339840:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189400.764760:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189400.764786:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a0146fc7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664340288:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189400.764808:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189400.764840:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664340352 00000100:00100000:3.0:1776189400.764882:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8563 00000100:00100000:3.0:1776189400.764887:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000060a8344d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664340352:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189400.764944:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000060a8344d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664340352:12345-0@lo:400:kworker/u8:0.0 Request processed in 56us (232us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189400.764952:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8563 00000100:00100000:3.0:1776189400.764977:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664339968 00000100:00100000:3.0:1776189400.764998:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664339840 00000100:00100000:3.0:1776189400.765014:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664340288 00000100:00100000:3.0:1776189400.765028:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9965 00000100:00100000:3.0:1776189400.765032:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000031aca57e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664339968:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189400.765059:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000031aca57e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664339968:12345-0@lo:400:kworker/u8:0.0 Request processed in 27us (400us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189400.765065:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9965 00000100:00100000:3.0:1776189400.765077:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9966 00000100:00100000:3.0:1776189400.765080:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002460f3d1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664339840:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189400.765104:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002460f3d1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664339840:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (344us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189400.765109:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9966 00000100:00100000:3.0:1776189400.765120:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9967 00000100:00100000:3.0:1776189400.765123:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a50d0fa6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664340288:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189400.765146:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a50d0fa6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664340288:12345-0@lo:400:kworker/u8:0.0 Request processed in 22us (338us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189400.765152:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9967 00000100:00100000:3.0:1776189400.765214:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002e4c5085 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664339840:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189400.765224:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a0146fc7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664340288:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189400.765232:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008cc6ee78 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664339968:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189400.765239:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003b5d789a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664340352:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189400.765412:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fd39f2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664340160:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189400.765443:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189400.765466:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b45ebec6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664340416:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189400.765486:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189400.765500:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1776189400.765506:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000048520f37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664339776:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189400.765530:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189400.765543:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c3b654b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664340224:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189400.765563:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189400.765625:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664340160 00000100:00100000:2.0:1776189400.765813:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6665a35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664339904:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189400.765840:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189400.765855:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000822c622a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664340032:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189400.765873:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189400.765883:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c76be6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664340096:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189400.765905:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189400.765945:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664339904 00000100:00100000:2.0:1776189400.765961:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664340032 00000100:00100000:2.0:1776189400.765969:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57377 00000100:00100000:2.0:1776189400.765973:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fb1c1d52 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664339904:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189400.765987:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189400.766018:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fb1c1d52 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664339904:12345-0@lo:38:kworker/u8:0.0 Request processed in 45us (178us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189400.766024:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57377 00000100:00100000:2.0:1776189400.766036:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57378 00000100:00100000:2.0:1776189400.766039:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d1c360e6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664340032:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189400.766045:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189400.766065:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d1c360e6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664340032:12345-0@lo:38:kworker/u8:0.0 Request processed in 25us (192us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189400.766070:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57378 00000100:00100000:2.0:1776189400.766119:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664340096 00000100:00100000:2.0:1776189400.766136:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9125 00000100:00100000:2.0:1776189400.766140:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000075ec22ba pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664340096:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189400.766146:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189400.766168:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000075ec22ba pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664340096:12345-0@lo:38:kworker/u8:0.0 Request processed in 27us (263us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189400.766190:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9125 00000100:00100000:2.0:1776189400.766387:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664339776 00000100:00100000:2.0:1776189400.766411:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1461 00000100:00100000:2.0:1776189400.766415:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a9f52f65 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664339776:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189400.766443:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a9f52f65 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664339776:12345-0@lo:400:kworker/u8:0.0 Request processed in 27us (913us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189400.766449:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1461 00000100:00100000:2.0:1776189400.766474:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000031b5a624 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664340480:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189400.766498:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1776189400.766535:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189400.766538:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189400.766542:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6665a35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664339904:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776189400.766549:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189400.766551:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189400.766554:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000822c622a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664340032:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776189400.766560:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189400.766563:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189400.766565:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c76be6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664340096:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189400.766727:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664340416 00000100:00100000:0.0:1776189400.770554:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664340224 00000100:00100000:0.0:1776189400.770626:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8564 00000100:00100000:0.0:1776189400.770634:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000476e266d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664340160:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189400.770733:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000476e266d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664340160:12345-0@lo:400:kworker/u8:0.0 Request processed in 100us (5290us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189400.770741:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8564 00000100:00100000:0.0:1776189400.770759:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8565 00000100:00100000:0.0:1776189400.770762:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053c82750 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664340416:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189400.770800:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053c82750 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664340416:12345-0@lo:400:kworker/u8:0.0 Request processed in 37us (5314us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189400.770809:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8565 00000100:00100000:0.0:1776189400.770821:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8566 00000100:00100000:0.0:1776189400.770824:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004920d098 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664340224:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189400.770846:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004920d098 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664340224:12345-0@lo:400:kworker/u8:0.0 Request processed in 21us (5283us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189400.770851:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8566 00000100:00100000:0.0:1776189400.770927:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664340480 00000100:00100000:0.0:1776189400.770951:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6806 00000100:00100000:0.0:1776189400.770954:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002ed2a919 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664340480:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776189400.770967:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189400.770999:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002ed2a919 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664340480:12345-0@lo:13:osp-pre-0-1.0 Request processed in 44us (4501us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189400.771005:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6806 00000100:00100000:0.0:1776189400.771058:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000025f5718c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664340544:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189400.771089:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189400.771118:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664340544 00000100:00100000:0.0:1776189400.771136:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6807 00000100:00100000:0.0:1776189400.771139:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0895cec pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664340544:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189400.771148:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189400.771191:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0895cec pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664340544:12345-0@lo:13:osp-pre-1-1.0 Request processed in 51us (102us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189400.771197:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6807 00000100:00100000:0.0:1776189400.771233:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000025f5718c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664340544:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189400.771268:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000031b5a624 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664340480:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189400.771293:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000048520f37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664339776:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189400.771304:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c3b654b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664340224:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189400.771314:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fd39f2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664340160:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189400.771322:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b45ebec6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664340416:0@lo:400:kworker/u8:0.0 00010000:02020000:2.0:1776189400.796310:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189401.053003:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189401.084479:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189401.309120:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189401.564283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189401.820282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189402.076329:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189402.109088:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189402.332332:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189402.588347:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189402.845087:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189403.100472:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189403.134041:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189403.357687:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189403.613158:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189403.868640:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189404.124290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189404.156658:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189404.380494:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189404.540321:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189404.540376:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189404.540453:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189404.540554:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654954240 00000100:00100000:1.0:1776189404.540609:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1462 00000100:00100000:1.0:1776189404.540615:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000be36d03e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654954240:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776189404.540682:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000be36d03e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654954240:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 67us (306us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189404.540690:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1462 00000100:00100000:2.0:1776189404.540865:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654954304 00000100:00100000:2.0:1776189404.540902:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654954368 00000100:00100000:2.0:1776189404.540920:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9968 00000100:00100000:2.0:1776189404.540925:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000025f5718c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654954304:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776189404.540958:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000025f5718c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654954304:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 32us (645us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189404.540964:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9968 00000100:00100000:2.0:1776189404.540976:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57379 00000100:00100000:2.0:1776189404.540980:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a1abf153 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654954368:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:2.0:1776189404.540993:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189404.541017:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a1abf153 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654954368:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 36us (564us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189404.541022:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57379 00010000:02020000:2.0:1776189404.636622:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189404.861265:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f3cbce47 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664340608:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189404.861340:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189404.862068:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664340608 00000100:00100000:1.0:1776189404.862117:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6808 00000100:00100000:1.0:1776189404.862123:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002b22e0b2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664340608:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189404.862155:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189404.862235:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002b22e0b2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664340608:12345-0@lo:13:osp-pre-1-0.0 Request processed in 111us (896us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189404.862241:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6808 00000100:00100000:3.0:1776189404.863604:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f3cbce47 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664340608:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189404.863784:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776189404.863797:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cbabaed7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664340672:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189404.863858:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189404.864216:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664340672 00000100:00100000:0.0:1776189404.864263:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6809 00000100:00100000:0.0:1776189404.864269:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008e2382dc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664340672:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776189404.864305:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189404.864353:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008e2382dc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664340672:12345-0@lo:13:osp-pre-0-0.0 Request processed in 84us (495us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189404.864359:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6809 00000100:00100000:3.0:1776189404.864482:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cbabaed7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664340672:0@lo:13:osp-pre-0-0.0 00010000:02020000:2.0:1776189404.892287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189405.148294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189405.180773:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189405.404410:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189405.660371:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189405.884404:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776189405.884427:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000412a95e8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664340736:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189405.884492:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189405.885356:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664340736 00000100:00100000:0.0:1776189405.885411:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6810 00000100:00100000:0.0:1776189405.885417:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000074a40efb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664340736:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776189405.885441:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189405.885493:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000074a40efb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664340736:12345-0@lo:13:osp-pre-0-1.0 Request processed in 75us (1001us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189405.885500:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6810 00000100:00100000:2.0:1776189405.886820:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000095a4d23a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664340800:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189405.886870:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189405.887129:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664340800 00000100:00100000:1.0:1776189405.887204:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6811 00000100:00100000:1.0:1776189405.887209:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d9eb44b4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664340800:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189405.887226:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189405.887264:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d9eb44b4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664340800:12345-0@lo:13:osp-pre-1-1.0 Request processed in 54us (394us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189405.887270:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6811 00000100:00080000:2.0:1776189405.887326:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189405.887330:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189405.887333:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189405.887337:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9547 00000100:00080000:2.0:1776189405.887340:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00080000:2.0:1776189405.887344:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189405.887362:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000e15ac1c x1862459664340992/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189405.887383:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189405.887386:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189405.887387:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189405.887391:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9547 00000100:00080000:2.0:1776189405.887392:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:2.0:1776189405.887395:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189405.887404:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f078b3a8 x1862459664341120/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189405.887412:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189405.887414:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189405.887415:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189405.887418:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9547 00000100:00080000:2.0:1776189405.887420:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 11s 00000100:00080000:2.0:1776189405.887422:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189405.887432:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e42142fc x1862459664341184/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189405.887465:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e0bab4ab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664341248:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189405.887496:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189405.887507:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000089232900 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664341056:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189405.887523:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000095a4d23a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664340800:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189405.887536:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189405.887542:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1776189405.887546:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001706ae06 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664340864:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189405.887552:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1776189405.887555:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000895d417a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664340928:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189405.887569:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189405.887574:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189405.887583:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9c407a2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664341312:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189405.887584:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d729b5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664341376:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189405.887601:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189405.887601:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189405.887622:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664340864 00000100:00100000:3.0:1776189405.887622:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664341056 00000100:00100000:2.0:1776189405.887645:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1463 00000100:00100000:3.0:1776189405.887646:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664340928 00000100:00100000:2.0:1776189405.887649:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000816fca97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664340864:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189405.887660:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664341376 00000100:00100000:2.0:1776189405.887692:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000816fca97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664340864:12345-0@lo:400:kworker/u8:1.0 Request processed in 42us (122us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189405.887697:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1463 00000100:00100000:2.0:1776189405.887718:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e15ac1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664340992:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189405.887737:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189405.887748:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f078b3a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664341120:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189405.887753:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664340992 00000100:00100000:3.0:1776189405.887764:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9969 00000100:00100000:2.0:1776189405.887766:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189405.887768:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003b5d789a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664341056:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189405.887772:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e42142fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664341184:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189405.887798:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003b5d789a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664341056:12345-0@lo:400:kworker/u8:1.0 Request processed in 29us (261us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189405.887802:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189405.887804:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9969 00000100:00100000:3.0:1776189405.887817:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664341120 00000100:00100000:2.0:1776189405.887824:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664341184 00000100:00100000:3.0:1776189405.887826:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9970 00000100:00100000:3.0:1776189405.887829:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008cc6ee78 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664340928:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189405.887839:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9126 00000100:00100000:2.0:1776189405.887841:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000008468a4e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664341184:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189405.887855:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189405.887865:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008cc6ee78 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664340928:12345-0@lo:400:kworker/u8:1.0 Request processed in 36us (291us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189405.887871:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9970 00000100:00100000:2.0:1776189405.887880:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000008468a4e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664341184:12345-0@lo:38:kworker/u8:1.0 Request processed in 38us (78us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189405.887882:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9971 00000100:00100000:3.0:1776189405.887885:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0146fc7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664341376:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189405.887896:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9126 00000100:00100000:3.0:1776189405.887909:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0146fc7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664341376:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (308us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189405.887914:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57380 00000100:00100000:2.0:1776189405.887917:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000af79f780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664340992:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189405.887924:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189405.887945:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000af79f780 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664340992:12345-0@lo:38:kworker/u8:1.0 Request processed in 27us (208us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189405.887950:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000412a95e8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664340736:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189405.888024:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9971 00000100:00100000:3.0:1776189405.888037:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57381 00000100:00100000:3.0:1776189405.888040:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ae28b57d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664341120:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189405.888047:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189405.888069:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ae28b57d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664341120:12345-0@lo:38:kworker/u8:1.0 Request processed in 28us (303us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189405.888074:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57381 00000100:00100000:3.0:1776189405.888096:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000895d417a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664340928:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189405.888103:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d729b5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664341376:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189405.888112:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000089232900 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664341056:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189405.888144:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664341248 00000100:00100000:3.0:1776189405.888196:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664341312 00000100:00100000:3.0:1776189405.888215:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8567 00000100:00100000:3.0:1776189405.888219:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000733a9e44 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664341248:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189405.888245:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000733a9e44 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664341248:12345-0@lo:400:kworker/u8:1.0 Request processed in 26us (749us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189405.888251:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8567 00000100:00100000:3.0:1776189405.888261:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8568 00000100:00100000:3.0:1776189405.888264:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000748b663e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664341312:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189405.888286:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000748b663e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664341312:12345-0@lo:400:kworker/u8:1.0 Request processed in 21us (686us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189405.889478:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57380 00000100:00080000:2.0:1776189405.889531:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189405.889535:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189405.889538:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e15ac1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664340992:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776189405.889545:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189405.889548:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189405.889549:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f078b3a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664341120:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776189405.889557:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189405.889559:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189405.889560:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e42142fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664341184:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189405.889604:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001706ae06 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664340864:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189405.889613:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9c407a2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664341312:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189405.889622:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e0bab4ab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664341248:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189405.889856:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8568 00010000:02020000:2.0:1776189405.916786:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189406.172306:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189406.205451:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189406.428317:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189406.684389:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189406.940622:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189407.196247:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189407.229131:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189407.452288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189407.708309:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189407.964423:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189408.225286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189408.252878:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189408.476936:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189408.732356:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189408.988286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189409.244301:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189409.276732:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189409.500853:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189409.658290:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189409.658343:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189409.658498:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654954496 00000100:00100000:3.0:1776189409.658621:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9972 00000100:00100000:3.0:1776189409.658628:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001264a122 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654954496:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189409.658697:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001264a122 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654954496:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 69us (354us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189409.658704:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9972 00000100:00100000:3.0:1776189409.658791:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189409.658855:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654954560 00000100:00100000:3.0:1776189409.658875:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57382 00000100:00100000:3.0:1776189409.658879:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002e4c5085 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654954560:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:3.0:1776189409.658895:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776189409.659112:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002e4c5085 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654954560:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 49us (138us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189409.659120:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57382 00000100:00100000:1.0:1776189409.659549:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654954432 00000100:00100000:1.0:1776189409.659605:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1464 00000100:00100000:1.0:1776189409.659610:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c4f71e57 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654954432:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189409.659647:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c4f71e57 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654954432:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 36us (1365us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189409.659653:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1464 00010000:02020000:0.0:1776189409.756490:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189409.982205:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003f67f1c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664341440:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189409.982268:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189409.982420:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664341440 00000100:00100000:1.0:1776189409.982471:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6812 00000100:00100000:1.0:1776189409.982477:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005bd91e44 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664341440:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189409.982504:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189409.982553:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005bd91e44 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664341440:12345-0@lo:13:osp-pre-1-0.0 Request processed in 75us (285us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189409.982570:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6812 00000100:00100000:3.0:1776189409.982837:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003f67f1c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664341440:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189409.983895:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000403a47d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664341504:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189409.983945:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189409.987061:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664341504 00000100:00100000:1.0:1776189409.987136:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6813 00000100:00100000:1.0:1776189409.987142:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e0bab4ab pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664341504:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189409.987194:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189409.987277:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e0bab4ab pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664341504:12345-0@lo:13:osp-pre-0-0.0 Request processed in 134us (3332us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189409.987285:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6813 00000100:00100000:0.0:1776189409.987486:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000403a47d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664341504:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189410.012313:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189410.268262:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189410.300565:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189410.524290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189410.780302:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189411.004364:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776189411.004389:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003f67f1c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664341568:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189411.004455:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189411.004540:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664341568 00000100:00100000:1.0:1776189411.004587:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6814 00000100:00080000:3.0:1776189411.004592:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1776189411.004593:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e9c407a2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664341568:12345-0@lo:13:osp-pre-1-1.0 00000100:00080000:3.0:1776189411.004597:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189411.004600:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189411.004606:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9552 00000100:00080000:3.0:1776189411.004609:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1776189411.004613:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00002000:00000024:1.0:1776189411.004625:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189411.004629:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007fd39f2e x1862459664341760/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189411.004650:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189411.004653:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189411.004654:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189411.004657:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9552 00000100:00100000:1.0:1776189411.004658:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e9c407a2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664341568:12345-0@lo:13:osp-pre-1-1.0 Request processed in 65us (204us total) trans 0 rc 0/0 00000100:00080000:3.0:1776189411.004659:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1776189411.004662:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189411.004664:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6814 00000100:00100000:3.0:1776189411.004670:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000048520f37 x1862459664341888/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189411.004677:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189411.004679:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189411.004680:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189411.004684:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9552 00000100:00080000:3.0:1776189411.004686:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1776189411.004689:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189411.004698:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b45ebec6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664341696:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189411.004698:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c76be6ad x1862459664341952/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189411.004722:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189411.004739:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bfa9efc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664342144:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189411.004772:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189411.004777:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fd39f2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664341760:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189411.004811:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000db3523c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664341632:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189411.004886:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000822c622a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664342016:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189411.005145:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664341696 00000100:00100000:3.0:1776189411.005191:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664342144 00000100:00100000:3.0:1776189411.005208:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9973 00000100:00100000:3.0:1776189411.005212:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000095a4d23a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664341696:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189411.005318:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9974 00000100:00100000:1.0:1776189411.005323:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f3cbce47 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664342144:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189411.005356:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f3cbce47 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664342144:12345-0@lo:400:kworker/u8:0.0 Request processed in 33us (584us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189411.005362:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9974 00000100:00100000:1.0:1776189411.005381:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189411.005393:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000048520f37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664341888:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189411.005414:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189411.005440:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c76be6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664341952:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189411.005460:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189411.005484:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664341760 00000100:00100000:1.0:1776189411.005500:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664341888 00000100:00100000:1.0:1776189411.005509:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57383 00000100:00100000:1.0:1776189411.005512:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000694c72a3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664341760:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189411.005525:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189411.005553:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000694c72a3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664341760:12345-0@lo:38:kworker/u8:0.0 Request processed in 39us (171us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189411.005568:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57383 00000100:00100000:1.0:1776189411.005578:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57384 00000100:00100000:1.0:1776189411.005581:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d4966c1b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664341888:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189411.005588:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189411.005730:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d4966c1b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664341888:12345-0@lo:38:kworker/u8:0.0 Request processed in 147us (315us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189411.005736:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57384 00000100:00100000:1.0:1776189411.005766:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c3b654b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664341824:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189411.005787:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189411.005799:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001b0eae72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664342208:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189411.005817:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189411.005849:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003f67f1c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664341568:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189411.005864:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776189411.005868:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba280200 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664342272:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189411.005886:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189411.005908:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664342208 00000100:00100000:1.0:1776189411.005933:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8569 00000100:00100000:1.0:1776189411.005937:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a3202503 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664342208:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189411.005968:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a3202503 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664342208:12345-0@lo:400:kworker/u8:0.0 Request processed in 31us (151us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189411.005974:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8569 00000100:00100000:1.0:1776189411.005988:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664342272 00000100:00100000:1.0:1776189411.006005:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6815 00000100:00100000:1.0:1776189411.006008:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003f67f1c5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664342272:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189411.006019:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189411.006042:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003f67f1c5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664342272:12345-0@lo:13:osp-pre-0-1.0 Request processed in 34us (156us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189411.006047:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6815 00000100:00080000:1.0:1776189411.006066:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189411.006069:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189411.006072:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fd39f2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664341760:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776189411.006079:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189411.006082:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189411.006084:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000048520f37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664341888:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189411.006109:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bfa9efc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664342144:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189411.006123:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664341824 00000100:00100000:1.0:1776189411.006136:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9975 00000100:00100000:1.0:1776189411.006139:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000631ae276 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664341824:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189411.006164:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000631ae276 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664341824:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (377us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189411.006169:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9975 00000100:00100000:1.0:1776189411.006211:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba280200 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664342272:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189411.006219:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c3b654b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664341824:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189411.006225:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001b0eae72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664342208:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189411.006338:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664341952 00000100:00100000:1.0:1776189411.006356:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9127 00000100:00100000:1.0:1776189411.006359:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008a7066b0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664341952:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189411.006368:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189411.006395:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008a7066b0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664341952:12345-0@lo:38:kworker/u8:0.0 Request processed in 34us (934us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189411.006400:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9127 00000100:00080000:1.0:1776189411.006434:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189411.006436:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189411.006439:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c76be6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664341952:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189411.006656:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189411.006681:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6665a35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664342080:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189411.006704:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189411.006728:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664342080 00000100:00100000:2.0:1776189411.006746:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8570 00000100:00100000:2.0:1776189411.006750:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001706ae06 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664342080:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189411.006779:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001706ae06 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664342080:12345-0@lo:400:kworker/u8:0.0 Request processed in 29us (75us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189411.006785:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8570 00000100:00100000:2.0:1776189411.006806:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6665a35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664342080:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189411.011121:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664341632 00000100:00100000:1.0:1776189411.011223:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1465 00000100:00100000:1.0:1776189411.011229:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b1a6ecdc pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664341632:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189411.011318:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b1a6ecdc pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664341632:12345-0@lo:400:kworker/u8:0.0 Request processed in 88us (4662us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189411.011325:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1465 00000100:00100000:0.0:1776189411.011825:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189411.011916:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664342016 00000100:00100000:1.0:1776189411.011953:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8571 00000100:00100000:1.0:1776189411.011957:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000403a47d8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664342016:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189411.011994:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000403a47d8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664342016:12345-0@lo:400:kworker/u8:0.0 Request processed in 36us (172us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189411.011999:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8571 00000100:00100000:0.0:1776189411.012272:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000822c622a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664342016:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189411.012913:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000db3523c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664341632:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189411.013366:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000095a4d23a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664341696:12345-0@lo:400:kworker/u8:0.0 Request processed in 8152us (8642us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189411.013375:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9973 00000100:00100000:1.0:1776189411.013892:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b45ebec6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664341696:0@lo:400:kworker/u8:0.0 00010000:02020000:0.0:1776189411.038267:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189411.294203:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189411.324383:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189411.548382:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189411.807393:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189412.060299:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189412.316378:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189412.348799:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189412.572357:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189412.828307:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189413.084518:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189413.340316:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189413.372664:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189413.596353:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189413.852439:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189414.108435:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189414.364383:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189414.396550:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189414.620952:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189414.781792:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189414.781888:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189414.783250:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654954688 00000100:00100000:0.0:1776189414.783315:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9976 00000100:00100000:0.0:1776189414.783322:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000aa3f364b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654954688:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189414.783413:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000aa3f364b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654954688:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 91us (1525us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189414.783430:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9976 00000100:00100000:3.0:1776189414.783758:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189414.783858:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654954752 00000100:00100000:1.0:1776189414.783892:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57385 00000100:00100000:1.0:1776189414.783896:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008573eccf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654954752:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776189414.783911:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189414.783950:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008573eccf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654954752:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 52us (194us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189414.783957:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57385 00000100:00100000:2.0:1776189414.784198:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654954624 00000100:00100000:2.0:1776189414.784264:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1466 00000100:00100000:2.0:1776189414.784269:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000090683405 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654954624:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189414.784308:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000090683405 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654954624:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 38us (2524us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189414.784314:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1466 00010000:02020000:0.0:1776189414.876290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189415.100647:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba280200 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664342336:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189415.100717:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189415.100751:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000be3c8080 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664342400:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189415.100780:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664342336 00000100:00100000:3.0:1776189415.100791:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189415.100826:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664342400 00000100:00100000:2.0:1776189415.100853:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6816 00000100:00100000:2.0:1776189415.100861:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e42142fc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664342336:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189415.100893:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189415.100953:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e42142fc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664342336:12345-0@lo:13:osp-pre-1-0.0 Request processed in 92us (235us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189415.100964:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6816 00000100:00100000:2.0:1776189415.100983:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6817 00000100:00100000:2.0:1776189415.100987:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002ef57fef pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664342400:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189415.101003:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189415.103874:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba280200 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664342336:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189415.108058:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002ef57fef pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664342400:12345-0@lo:13:osp-pre-0-0.0 Request processed in 7060us (7258us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189415.108075:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6817 00000100:00100000:3.0:1776189415.117336:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000be3c8080 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664342400:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189415.132283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189415.388292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189415.420387:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189415.645772:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189415.900286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:2.0:1776189416.124322:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189416.124330:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189416.124333:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189416.124341:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9557 00000100:00080000:2.0:1776189416.124346:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776189416.124351:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189416.124372:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000412a95e8 x1862459664342592/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189416.124413:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189416.124417:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189416.124418:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189416.124421:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9557 00000100:00080000:2.0:1776189416.124423:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776189416.124427:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189416.124437:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d729b5cd x1862459664342720/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189416.124507:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189416.124511:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189416.124512:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189416.124515:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9557 00000100:00080000:2.0:1776189416.124517:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776189416.124521:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189416.124533:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000895d417a x1862459664342784/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189416.124548:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e15ac1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664342528:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189416.124613:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189416.124633:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000412a95e8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664342592:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189416.124669:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000031b5a624 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664342848:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189416.124694:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189416.124761:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664342528 00000100:00100000:2.0:1776189416.124806:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9977 00000100:00100000:2.0:1776189416.124811:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e6117432 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664342528:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189416.124898:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f078b3a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664342464:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189416.125289:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d4da5ec0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664343104:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189416.125392:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189416.125417:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1776189416.125423:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000089232900 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664342656:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189416.125545:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189416.125563:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000264516d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664343040:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189416.125586:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189416.125619:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664342848 00000100:00100000:3.0:1776189416.125737:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664343104 00000100:00100000:3.0:1776189416.125931:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664343040 00000100:00100000:3.0:1776189416.125951:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8572 00000100:00100000:3.0:1776189416.125956:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a829435f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664342848:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189416.126000:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a829435f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664342848:12345-0@lo:400:kworker/u8:1.0 Request processed in 43us (1306us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189416.126007:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8572 00000100:00100000:3.0:1776189416.126024:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8573 00000100:00100000:3.0:1776189416.126028:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008b54d6c4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664343104:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189416.126054:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008b54d6c4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664343104:12345-0@lo:400:kworker/u8:1.0 Request processed in 25us (736us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189416.126059:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8573 00000100:00100000:3.0:1776189416.126071:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8574 00000100:00100000:3.0:1776189416.126074:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ae1dab54 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664343040:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189416.126099:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ae1dab54 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664343040:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (512us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189416.126104:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8574 00000100:00100000:3.0:1776189416.126130:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189416.126150:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d729b5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664342720:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189416.126195:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189416.126294:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000895d417a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664342784:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189416.126402:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189416.126463:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664342656 00000100:00100000:3.0:1776189416.126488:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664342592 00000100:00100000:3.0:1776189416.126497:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664342720 00000100:00100000:3.0:1776189416.126505:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9978 00000100:00100000:3.0:1776189416.126508:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001b0eae72 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664342656:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189416.126539:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001b0eae72 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664342656:12345-0@lo:400:kworker/u8:1.0 Request processed in 30us (996us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189416.126545:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9978 00000100:00100000:3.0:1776189416.126558:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57386 00000100:00100000:3.0:1776189416.126561:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000066892405 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664342592:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189416.126574:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189416.126604:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000066892405 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664342592:12345-0@lo:38:kworker/u8:1.0 Request processed in 41us (473us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189416.126609:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57386 00000100:00100000:3.0:1776189416.126619:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57387 00000100:00100000:3.0:1776189416.126621:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000040cb0cca pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664342720:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189416.126627:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189416.126919:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001f4e478a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664342976:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189416.126949:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189416.127034:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000040cb0cca pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664342720:12345-0@lo:38:kworker/u8:1.0 Request processed in 410us (838us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189416.127041:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57387 00000100:00100000:3.0:1776189416.127053:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664342976 00000100:00100000:3.0:1776189416.127067:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9979 00000100:00100000:3.0:1776189416.127071:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000035725ab pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664342976:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189416.127308:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000035725ab pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664342976:12345-0@lo:400:kworker/u8:1.0 Request processed in 234us (357us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189416.127331:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d9b9c94e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664343232:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189416.127359:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189416.127379:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9979 00000100:00100000:0.0:1776189416.127392:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001f4e478a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664342976:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189416.127405:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664342784 00000100:00100000:0.0:1776189416.127411:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776189416.127418:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000266f5a0c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664343168:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189416.127425:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9128 00000100:00100000:3.0:1776189416.127430:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000013c0eaff pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664342784:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189416.127525:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189416.127552:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000013c0eaff pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664342784:12345-0@lo:38:kworker/u8:1.0 Request processed in 122us (1151us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189416.127558:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9128 00000100:00100000:0.0:1776189416.127579:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1776189416.127589:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189416.127593:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189416.127598:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000412a95e8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664342592:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776189416.127610:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189416.127613:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189416.127615:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d729b5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664342720:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189416.127616:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664343232 00000100:00080000:3.0:1776189416.127625:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189416.127628:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189416.127631:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000895d417a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664342784:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189416.127644:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664343168 00000100:00100000:0.0:1776189416.127661:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6818 00000100:00100000:0.0:1776189416.127665:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ace99b9b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664343232:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776189416.127680:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189416.127716:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ace99b9b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664343232:12345-0@lo:13:osp-pre-0-1.0 Request processed in 49us (356us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189416.127722:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6818 00000100:00100000:0.0:1776189416.127737:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6819 00000100:00100000:0.0:1776189416.127743:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000014a0e5e7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664343168:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189416.127753:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189416.127778:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000014a0e5e7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664343168:12345-0@lo:13:osp-pre-1-1.0 Request processed in 36us (202us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189416.127784:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6819 00000100:00100000:0.0:1776189416.127818:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000266f5a0c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664343168:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189416.127828:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d9b9c94e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664343232:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189416.127957:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000089232900 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664342656:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189416.127972:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000264516d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664343040:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189416.127981:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000031b5a624 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664342848:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189416.127989:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d4da5ec0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664343104:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189416.128113:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e6117432 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664342528:12345-0@lo:400:kworker/u8:1.0 Request processed in 3300us (3500us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189416.128121:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9977 00000100:00100000:3.0:1776189416.128202:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e15ac1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664342528:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189416.128530:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189416.128558:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000adb1e111 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664342912:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189416.128586:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189416.128787:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664342464 00000100:00100000:2.0:1776189416.128826:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1467 00000100:00100000:2.0:1776189416.128830:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b90a26ba pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664342464:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189416.128876:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b90a26ba pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664342464:12345-0@lo:400:kworker/u8:1.0 Request processed in 44us (440us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189416.128882:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1467 00000100:00100000:3.0:1776189416.129076:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f078b3a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664342464:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189416.129123:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664342912 00000100:00100000:0.0:1776189416.129221:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8575 00000100:00100000:0.0:1776189416.129225:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba280200 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664342912:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189416.129270:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba280200 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664342912:12345-0@lo:400:kworker/u8:1.0 Request processed in 44us (684us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189416.129276:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8575 00000100:00100000:3.0:1776189416.129340:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000adb1e111 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664342912:0@lo:400:kworker/u8:1.0 00010000:02020000:0.0:1776189416.157035:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189416.412862:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189416.444502:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189416.668529:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189416.924349:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189417.180307:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189417.436338:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189417.469463:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189417.692510:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189417.949170:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189418.204752:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189418.460290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189418.492501:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189418.716611:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189418.972683:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189419.231281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189419.485790:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189419.516397:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189419.740293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189419.901674:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189419.901725:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189419.901802:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189419.901892:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654954880 00000100:00100000:0.0:1776189419.901956:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654954944 00000100:00100000:0.0:1776189419.901977:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9980 00000100:00100000:0.0:1776189419.901983:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b45ebec6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654954880:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189419.902033:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57388 00000100:00100000:2.0:1776189419.902039:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000db3523c9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654954944:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776189419.902053:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189419.902084:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000db3523c9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654954944:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 44us (281us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189419.902092:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57388 00000100:00100000:0.0:1776189419.902359:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b45ebec6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654954880:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 71us (391us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189419.902376:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9980 00000100:00100000:3.0:1776189419.902425:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654954816 00000100:00100000:3.0:1776189419.902471:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1468 00000100:00100000:3.0:1776189419.902475:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ef3ea18a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654954816:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189419.902517:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ef3ea18a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654954816:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 42us (793us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189419.902523:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1468 00010000:02020000:0.0:1776189419.996255:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189420.220448:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000822c622a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664343296:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189420.220515:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189420.220619:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664343296 00000100:00100000:1.0:1776189420.220672:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6820 00000100:00100000:1.0:1776189420.220677:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d9b9c94e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664343296:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189420.220705:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189420.220760:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d9b9c94e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664343296:12345-0@lo:13:osp-pre-1-0.0 Request processed in 82us (245us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189420.220768:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6820 00000100:00100000:2.0:1776189420.220919:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000822c622a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664343296:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189420.221881:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776189420.221896:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6665a35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664343360:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189420.221938:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189420.222021:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664343360 00000100:00100000:0.0:1776189420.222077:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6821 00000100:00100000:0.0:1776189420.222082:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000aa8befb6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664343360:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776189420.222105:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189420.222150:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000aa8befb6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664343360:12345-0@lo:13:osp-pre-0-0.0 Request processed in 67us (212us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189420.222157:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6821 00000100:00100000:2.0:1776189420.222405:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6665a35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664343360:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189420.252494:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189420.508479:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189420.546141:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189420.764308:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189421.020284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189421.244412:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc84a765 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664343488:0@lo:13:osp-pre-1-1.0 00000100:00080000:0.0:1776189421.244446:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189421.244452:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189421.244455:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189421.244461:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9563 00000100:00080000:0.0:1776189421.244464:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1776189421.244470:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189421.244482:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189421.244491:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000056e9ad0b x1862459664343680/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189421.244518:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00080000:0.0:1776189421.244520:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189421.244525:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1776189421.244526:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d4da5ec0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664343424:0@lo:13:osp-pre-0-1.0 00000100:00080000:0.0:1776189421.244527:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189421.244534:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9563 00000100:00080000:0.0:1776189421.244536:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1776189421.244539:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189421.244566:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000099b8eb0d x1862459664343808/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189421.244570:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776189421.244578:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189421.244581:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189421.244583:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189421.244587:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9563 00000100:00100000:2.0:1776189421.244587:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00080000:0.0:1776189421.244591:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1776189421.244594:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189421.244597:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000023fe30ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664343744:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189421.244609:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000201d0f3c x1862459664343872/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189421.244627:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189421.244651:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ccb5377d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664343936:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189421.244715:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000266f5a0c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664343552:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189421.244750:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189421.244768:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004d2de401 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664344000:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189421.244801:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189421.244819:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1776189421.244834:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a43a3e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664343616:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189421.244856:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189421.244871:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000be3c8080 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664344064:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189421.244890:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189421.244921:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664343744 00000100:00100000:3.0:1776189421.244948:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000056e9ad0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664343680:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189421.244961:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664343616 00000100:00100000:1.0:1776189421.244982:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664344064 00000100:00100000:3.0:1776189421.244986:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189421.245004:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000099b8eb0d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664343808:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189421.245039:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189421.245044:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189421.245210:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664343488 00000100:00100000:0.0:1776189421.245236:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664343424 00000100:00100000:0.0:1776189421.245256:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6822 00000100:00100000:0.0:1776189421.245262:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d72c99d5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664343488:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189421.245283:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189421.245482:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d72c99d5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664343488:12345-0@lo:13:osp-pre-1-1.0 Request processed in 218us (1000us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189421.245489:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6822 00000100:00100000:0.0:1776189421.245505:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6823 00000100:00100000:0.0:1776189421.245510:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000032656100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664343424:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776189421.245520:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189421.245543:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000032656100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664343424:12345-0@lo:13:osp-pre-0-1.0 Request processed in 33us (977us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189421.245633:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6823 00000100:00100000:0.0:1776189421.245655:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664344000 00000100:00100000:0.0:1776189421.245676:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664343936 00000100:00100000:0.0:1776189421.245692:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8576 00000100:00100000:0.0:1776189421.245695:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c76be6ad pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664344000:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189421.245742:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c76be6ad pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664344000:12345-0@lo:400:kworker/u8:1.0 Request processed in 45us (941us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189421.245748:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8576 00000100:00100000:0.0:1776189421.245758:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8577 00000100:00100000:0.0:1776189421.245761:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000029eeaafe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664343936:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189421.245785:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000029eeaafe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664343936:12345-0@lo:400:kworker/u8:1.0 Request processed in 23us (748us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189421.245790:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8577 00000100:00100000:3.0:1776189421.246370:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000201d0f3c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664343872:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189421.247393:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664343680 00000100:00100000:1.0:1776189421.247429:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664343808 00000100:00100000:1.0:1776189421.247443:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9981 00000100:00100000:1.0:1776189421.247448:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f159c11f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664343744:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189421.247514:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f159c11f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664343744:12345-0@lo:400:kworker/u8:1.0 Request processed in 66us (2889us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189421.247521:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9981 00000100:00100000:1.0:1776189421.247549:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9982 00000100:00100000:1.0:1776189421.247553:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006c3b654b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664343616:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189421.247581:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006c3b654b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664343616:12345-0@lo:400:kworker/u8:1.0 Request processed in 27us (2725us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189421.247587:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9982 00000100:00100000:1.0:1776189421.247597:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9983 00000100:00100000:1.0:1776189421.247601:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000bfa9efc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664344064:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189421.247626:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000bfa9efc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664344064:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (2735us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189421.247631:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9983 00000100:00100000:1.0:1776189421.247643:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57389 00000100:00100000:1.0:1776189421.247646:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006a7d5f1c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664343680:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189421.247659:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189421.247697:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006a7d5f1c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664343680:12345-0@lo:38:kworker/u8:1.0 Request processed in 50us (2714us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189421.247703:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57389 00000100:00100000:1.0:1776189421.247714:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57390 00000100:00100000:1.0:1776189421.247718:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000051b95bd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664343808:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189421.247724:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189421.247744:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000051b95bd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664343808:12345-0@lo:38:kworker/u8:1.0 Request processed in 25us (2702us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189421.247749:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57390 00000100:00100000:1.0:1776189421.247785:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664343552 00000100:00100000:3.0:1776189421.248407:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189421.248442:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1469 00000100:00100000:1.0:1776189421.248447:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005c923f64 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664343552:12345-0@lo:400:kworker/u8:1.0 00000100:00080000:3.0:1776189421.248475:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189421.248480:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189421.248485:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000056e9ad0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664343680:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776189421.248496:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189421.248498:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189421.248501:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000099b8eb0d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664343808:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189421.248547:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664343872 00000100:00100000:1.0:1776189421.248591:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005c923f64 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664343552:12345-0@lo:400:kworker/u8:1.0 Request processed in 142us (3839us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189421.248598:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1469 00000100:00100000:1.0:1776189421.248640:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a43a3e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664343616:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189421.248645:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9129 00000100:00100000:3.0:1776189421.248652:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007ec15c7b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664343872:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189421.248657:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000be3c8080 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664344064:0@lo:400:kworker/u8:1.0 00010000:02000000:3.0:1776189421.248663:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189421.248670:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000266f5a0c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664343552:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189421.248773:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004d2de401 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664344000:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189421.249448:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007ec15c7b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664343872:12345-0@lo:38:kworker/u8:1.0 Request processed in 792us (1039us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189421.249458:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9129 00000100:00080000:2.0:1776189421.249467:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189421.249472:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189421.249476:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000201d0f3c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664343872:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189421.249519:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ccb5377d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664343936:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189421.249531:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000023fe30ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664343744:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189421.249548:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d4da5ec0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664343424:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189421.249562:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc84a765 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664343488:0@lo:13:osp-pre-1-1.0 00010000:02020000:0.0:1776189421.276507:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189421.532391:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189421.564461:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189421.789393:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189422.044325:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189422.300683:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189422.556405:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189422.589931:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189422.812316:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189423.068288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189423.324723:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189423.580376:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189423.615678:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189423.837309:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189424.092400:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189424.348754:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189424.604299:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189424.640021:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189424.860694:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189425.018976:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189425.019028:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189425.019110:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654955072 00000100:00100000:2.0:1776189425.019209:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9984 00000100:00100000:2.0:1776189425.019215:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000048520f37 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654955072:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776189425.019303:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000048520f37 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654955072:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 87us (334us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189425.019311:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9984 00000100:00100000:3.0:1776189425.020095:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654955008 00000100:00100000:3.0:1776189425.020285:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1470 00000100:00100000:3.0:1776189425.020290:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cd00b680 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654955008:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1776189425.020340:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cd00b680 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654955008:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 50us (1313us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189425.020347:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1470 00000100:00100000:1.0:1776189425.027729:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189425.028020:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654955136 00000100:00100000:2.0:1776189425.028075:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57391 00000100:00100000:2.0:1776189425.028082:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007fd39f2e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654955136:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:2.0:1776189425.028100:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189425.028207:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007fd39f2e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654955136:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 125us (485us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189425.028214:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57391 00010000:02020000:0.0:1776189425.119601:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189425.341083:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776189425.341107:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009ea5366f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664344128:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189425.341197:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189425.341258:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664344128 00000100:00100000:0.0:1776189425.341302:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6824 00000100:00100000:0.0:1776189425.341308:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000023fe30ce pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664344128:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189425.341338:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189425.341386:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000023fe30ce pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664344128:12345-0@lo:13:osp-pre-1-0.0 Request processed in 77us (213us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189425.341394:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6824 00000100:00100000:3.0:1776189425.341957:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009ea5366f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664344128:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189425.342104:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776189425.342115:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6665a35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664344192:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189425.342148:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189425.342331:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664344192 00000100:00100000:1.0:1776189425.342409:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6825 00000100:00100000:1.0:1776189425.342415:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ccb5377d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664344192:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189425.342445:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189425.342491:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ccb5377d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664344192:12345-0@lo:13:osp-pre-0-0.0 Request processed in 75us (343us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189425.342497:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6825 00000100:00100000:0.0:1776189425.342563:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6665a35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664344192:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189425.372295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189425.628330:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189425.660394:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189425.884390:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189426.140289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189426.366050:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cbabaed7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664344256:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189426.366118:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189426.366151:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776189426.366157:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000032f89d8f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664344320:0@lo:400:kworker/u8:1.0 00000100:00080000:3.0:1776189426.366212:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189426.366218:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189426.366221:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189426.366236:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9568 00000100:00100000:1.0:1776189426.366238:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1776189426.366241:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776189426.366247:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189426.366269:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f078b3a8 x1862459664344448/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189426.366295:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189426.366299:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189426.366300:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1776189426.366303:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664344256 00000100:00080000:3.0:1776189426.366304:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9568 00000100:00080000:3.0:1776189426.366308:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776189426.366312:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189426.366325:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000031b5a624 x1862459664344576/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189426.366333:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189426.366336:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189426.366337:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189426.366340:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9568 00000100:00080000:3.0:1776189426.366342:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776189426.366347:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189426.366348:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6826 00000100:00100000:1.0:1776189426.366356:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000822c622a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664344256:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189426.366361:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000089232900 x1862459664344640/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00002000:00000024:1.0:1776189426.366377:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189426.366416:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000822c622a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664344256:12345-0@lo:13:osp-pre-0-1.0 Request processed in 59us (298us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189426.366424:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6826 00000100:00100000:1.0:1776189426.366462:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000264516d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664344384:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189426.366491:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189426.366510:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776189426.366515:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e15ac1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664344512:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189426.366538:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189426.366571:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664344384 00000100:00100000:1.0:1776189426.366596:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664344512 00000100:00100000:1.0:1776189426.366612:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9985 00000100:00100000:1.0:1776189426.366616:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e6665a35 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664344384:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189426.366650:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e6665a35 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664344384:12345-0@lo:400:kworker/u8:1.0 Request processed in 33us (159us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189426.366656:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9985 00000100:00100000:1.0:1776189426.366668:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9986 00000100:00100000:1.0:1776189426.366671:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000032e99411 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664344512:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189426.366695:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000032e99411 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664344512:12345-0@lo:400:kworker/u8:1.0 Request processed in 23us (158us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189426.366701:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9986 00000100:00100000:1.0:1776189426.366731:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e15ac1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664344512:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189426.366741:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000264516d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664344384:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189426.366801:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009ea5366f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664344768:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189426.366826:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189426.369686:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000412a95e8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664344896:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189426.369743:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189426.369771:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776189426.369776:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d729b5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664344832:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189426.369797:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189426.369828:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664344896 00000100:00100000:3.0:1776189426.369863:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9987 00000100:00100000:3.0:1776189426.369868:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000e15ac1c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664344896:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189426.369920:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000e15ac1c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664344896:12345-0@lo:400:kworker/u8:1.0 Request processed in 52us (178us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189426.369927:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9987 00000100:00100000:3.0:1776189426.369964:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000412a95e8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664344896:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189426.369984:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664344832 00000100:00100000:3.0:1776189426.370003:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8578 00000100:00100000:3.0:1776189426.370017:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ab2fefce pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664344832:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189426.370283:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ab2fefce pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664344832:12345-0@lo:400:kworker/u8:1.0 Request processed in 274us (485us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189426.370289:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8578 00000100:00100000:3.0:1776189426.370318:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d729b5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664344832:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189426.370343:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000895d417a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664344704:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189426.370366:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189426.370397:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cbabaed7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664344256:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189426.370420:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664344704 00000100:00100000:3.0:1776189426.370439:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8579 00000100:00100000:3.0:1776189426.370442:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d0c1478c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664344704:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189426.370470:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d0c1478c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664344704:12345-0@lo:400:kworker/u8:1.0 Request processed in 27us (104us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189426.370476:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8579 00000100:00100000:3.0:1776189426.370496:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000895d417a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664344704:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189426.370510:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f078b3a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664344448:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189426.370532:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189426.370545:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000031b5a624 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664344576:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189426.370564:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189426.370575:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000089232900 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664344640:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189426.370596:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189426.370622:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664344640 00000100:00100000:3.0:1776189426.370644:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9130 00000100:00100000:3.0:1776189426.370647:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000932d0506 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664344640:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189426.370661:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189426.370691:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000932d0506 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664344640:12345-0@lo:38:kworker/u8:1.0 Request processed in 43us (94us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189426.370697:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9130 00000100:00100000:3.0:1776189426.370714:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664344448 00000100:00100000:3.0:1776189426.370731:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664344576 00000100:00100000:3.0:1776189426.370746:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57392 00000100:00100000:3.0:1776189426.370749:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f435eca pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664344448:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189426.370756:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189426.370779:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f435eca pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664344448:12345-0@lo:38:kworker/u8:1.0 Request processed in 28us (246us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189426.370784:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57392 00000100:00100000:3.0:1776189426.370794:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57393 00000100:00100000:3.0:1776189426.370798:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000239d0820 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664344576:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189426.370803:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189426.370822:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000239d0820 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664344576:12345-0@lo:38:kworker/u8:1.0 Request processed in 24us (258us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189426.370827:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57393 00000100:00080000:3.0:1776189426.370857:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189426.370861:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189426.370865:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f078b3a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664344448:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776189426.370872:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189426.370874:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189426.370876:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000031b5a624 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664344576:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776189426.370882:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189426.370884:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189426.370886:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000089232900 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664344640:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189426.371325:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664344768 00000100:00100000:3.0:1776189426.371353:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6827 00000100:00100000:3.0:1776189426.371356:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d52de5d9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664344768:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189426.371370:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189426.371405:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d52de5d9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664344768:12345-0@lo:13:osp-pre-1-1.0 Request processed in 48us (4579us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189426.371411:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6827 00000100:00100000:3.0:1776189426.371431:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664344320 00000100:00100000:3.0:1776189426.371454:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1471 00000100:00100000:3.0:1776189426.371458:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f6dc323d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664344320:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189426.371491:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f6dc323d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664344320:12345-0@lo:400:kworker/u8:1.0 Request processed in 32us (5269us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189426.371497:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1471 00000100:00100000:3.0:1776189426.371530:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000032f89d8f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664344320:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189426.373431:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009ea5366f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664344768:0@lo:13:osp-pre-1-1.0 00010000:02020000:0.0:1776189426.397454:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189426.652652:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189426.685846:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189426.908304:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189427.164306:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189427.420302:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189427.677783:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189427.708688:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189427.932516:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189428.188295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189428.444556:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189428.700443:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189428.732425:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189428.956293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189429.212568:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189429.468269:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189429.728275:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189429.756396:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189429.980284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189430.141276:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189430.141320:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189430.141360:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189430.141497:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654955200 00000100:00100000:1.0:1776189430.141550:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1472 00000100:00100000:1.0:1776189430.141557:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000078287bc4 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654955200:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776189430.141625:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000078287bc4 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654955200:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 68us (305us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189430.141633:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1472 00000100:00100000:2.0:1776189430.141940:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654955264 00000100:00100000:2.0:1776189430.141967:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9988 00000100:00100000:2.0:1776189430.141971:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fdab0c5c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654955264:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776189430.142000:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fdab0c5c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654955264:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 28us (734us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189430.142006:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9988 00000100:00100000:2.0:1776189430.142017:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654955328 00000100:00100000:2.0:1776189430.142029:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57394 00000100:00100000:2.0:1776189430.142033:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000eaf48489 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654955328:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:2.0:1776189430.142046:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189430.142068:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000eaf48489 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654955328:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 35us (709us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189430.142074:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57394 00010000:02020000:0.0:1776189430.236508:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189430.460422:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776189430.460445:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005be3c9f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664344960:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189430.460515:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189430.460674:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664344960 00000100:00100000:1.0:1776189430.460721:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6828 00000100:00100000:1.0:1776189430.460726:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cc84a765 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664344960:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189430.460759:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189430.460906:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776189430.460919:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005ad21a28 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664345024:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189430.465942:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cc84a765 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664344960:12345-0@lo:13:osp-pre-0-0.0 Request processed in 5208us (5420us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189430.465960:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6828 00000100:00100000:2.0:1776189430.466243:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005be3c9f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664344960:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189430.466373:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189430.466669:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664345024 00000100:00100000:1.0:1776189430.466717:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6829 00000100:00100000:1.0:1776189430.466723:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d45cc7cd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664345024:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189430.466752:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189430.467716:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d45cc7cd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664345024:12345-0@lo:13:osp-pre-1-0.0 Request processed in 986us (1339us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189430.467727:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6829 00000100:00100000:0.0:1776189430.469361:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005ad21a28 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664345024:0@lo:13:osp-pre-1-0.0 00010000:02020000:0.0:1776189430.492260:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189430.748358:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189430.780858:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189431.006034:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189431.260287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189431.484579:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009ea5366f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664345088:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189431.484648:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1776189431.484772:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189431.484777:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189431.484780:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189431.484785:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9573 00000100:00080000:2.0:1776189431.484789:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00080000:2.0:1776189431.484794:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189431.484816:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006047c8f7 x1862459664345280/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189431.484849:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189431.484852:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189431.484853:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189431.484856:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9573 00000100:00080000:2.0:1776189431.484858:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00080000:2.0:1776189431.484861:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189431.484870:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001aa39bf8 x1862459664345408/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189431.484878:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189431.484880:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189431.484882:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189431.484885:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9573 00000100:00080000:2.0:1776189431.484887:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 6s 00000100:00080000:2.0:1776189431.484889:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189431.484899:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e8ddaf27 x1862459664345472/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189431.484937:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003ff7f95c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664345344:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189431.484964:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189431.484982:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000075ad1ba3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664345728:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189431.485002:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189431.485020:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:2.0:1776189431.485024:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003da19f87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664345216:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189431.485043:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189431.485057:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d9a1a334 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664345664:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189431.485076:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189431.485096:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006047c8f7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664345280:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189431.485129:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189431.485142:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001aa39bf8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664345408:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189431.485161:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189431.485199:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e8ddaf27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664345472:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189431.485233:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189431.485266:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664345728 00000100:00100000:2.0:1776189431.485301:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8580 00000100:00100000:2.0:1776189431.485306:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000aecac7d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664345728:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189431.485350:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000aecac7d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664345728:12345-0@lo:400:kworker/u8:0.0 Request processed in 43us (347us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189431.485359:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8580 00000100:00100000:2.0:1776189431.485380:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664345344 00000100:00100000:2.0:1776189431.485399:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664345216 00000100:00100000:2.0:1776189431.485407:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9989 00000100:00100000:2.0:1776189431.485410:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003f21295b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664345344:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189431.485436:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003f21295b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664345344:12345-0@lo:400:kworker/u8:0.0 Request processed in 25us (472us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189431.485442:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9989 00000100:00100000:2.0:1776189431.485452:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664345664 00000100:00100000:2.0:1776189431.485462:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664345280 00000100:00100000:2.0:1776189431.485469:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664345408 00000100:00100000:2.0:1776189431.485477:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9990 00000100:00100000:2.0:1776189431.485481:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cda39ea2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664345216:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189431.485505:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cda39ea2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664345216:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (461us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189431.485511:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9990 00000100:00100000:2.0:1776189431.485522:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9991 00000100:00100000:2.0:1776189431.485525:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003a31bce8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664345664:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189431.485548:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003a31bce8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664345664:12345-0@lo:400:kworker/u8:0.0 Request processed in 22us (472us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189431.485553:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9991 00000100:00100000:2.0:1776189431.485562:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57395 00000100:00100000:2.0:1776189431.485565:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fd1686e0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664345280:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189431.485577:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189431.485603:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fd1686e0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664345280:12345-0@lo:38:kworker/u8:0.0 Request processed in 37us (475us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189431.485610:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57395 00000100:00100000:2.0:1776189431.485620:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57396 00000100:00100000:2.0:1776189431.485623:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008a791fe4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664345408:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189431.485628:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189431.485646:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008a791fe4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664345408:12345-0@lo:38:kworker/u8:0.0 Request processed in 22us (485us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189431.485651:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57396 00000100:00080000:2.0:1776189431.485692:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189431.485696:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189431.485700:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006047c8f7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664345280:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776189431.485707:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189431.485709:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189431.485711:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001aa39bf8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664345408:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189431.485740:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003da19f87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664345216:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189431.485748:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d9a1a334 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664345664:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189431.485756:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003ff7f95c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664345344:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189431.485763:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000075ad1ba3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664345728:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189431.486202:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664345472 00000100:00100000:3.0:1776189431.486244:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9131 00000100:00100000:3.0:1776189431.486248:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e6402bd2 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664345472:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189431.486256:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189431.486295:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e6402bd2 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664345472:12345-0@lo:38:kworker/u8:0.0 Request processed in 46us (1063us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189431.486302:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9131 00000100:00100000:3.0:1776189431.486330:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000048a53cb2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664345536:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189431.486356:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189431.486373:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1776189431.486378:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b696663d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664345152:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189431.486402:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189431.486414:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005c6b517b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664345600:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189431.486432:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189431.486454:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664345536 00000100:00100000:3.0:1776189431.486475:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664345600 00000100:00100000:3.0:1776189431.486489:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8581 00000100:00100000:3.0:1776189431.486493:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000064bc62e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664345536:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189431.486526:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000064bc62e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664345536:12345-0@lo:400:kworker/u8:0.0 Request processed in 33us (170us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189431.486532:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8581 00000100:00100000:3.0:1776189431.486545:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8582 00000100:00100000:3.0:1776189431.486548:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008950b44b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664345600:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189431.486570:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008950b44b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664345600:12345-0@lo:400:kworker/u8:0.0 Request processed in 21us (138us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189431.486575:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8582 00000100:00080000:3.0:1776189431.486598:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189431.486601:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189431.486604:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e8ddaf27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664345472:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189431.486628:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005c6b517b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664345600:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189431.486636:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000048a53cb2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664345536:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189431.486793:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664345088 00000100:00100000:1.0:1776189431.486829:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6830 00000100:00100000:1.0:1776189431.486834:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008758fb02 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664345088:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189431.486848:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189431.486878:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008758fb02 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664345088:12345-0@lo:13:osp-pre-1-1.0 Request processed in 44us (2231us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189431.486884:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6830 00000100:00100000:0.0:1776189431.487228:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776189431.487239:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d01df670 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664345792:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189431.487271:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189431.487309:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664345152 00000100:00100000:0.0:1776189431.487333:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1473 00000100:00100000:0.0:1776189431.487337:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003afac4ae pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664345152:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189431.487371:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003afac4ae pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664345152:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (970us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189431.487376:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1473 00000100:00100000:1.0:1776189431.487496:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664345792 00000100:00100000:1.0:1776189431.487523:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6831 00000100:00100000:1.0:1776189431.487526:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000088ec3431 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664345792:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189431.487536:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189431.487569:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000088ec3431 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664345792:12345-0@lo:13:osp-pre-0-1.0 Request processed in 42us (299us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189431.487575:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6831 00000100:00100000:1.0:1776189431.487607:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d01df670 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664345792:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189431.488083:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b696663d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664345152:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189431.488104:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009ea5366f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664345088:0@lo:13:osp-pre-1-1.0 00010000:02020000:0.0:1776189431.516287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189431.772310:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189431.805790:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189432.028292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189432.284242:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189432.540311:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189432.796512:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189432.829119:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189433.054448:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189433.308286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189433.565264:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189433.820310:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189433.852543:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189434.076461:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189434.333773:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189434.588362:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189434.844307:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189434.876740:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189435.100346:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189435.265673:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189435.265803:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189435.265885:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189435.265950:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654955392 00000100:00100000:2.0:1776189435.266013:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1474 00000100:00100000:2.0:1776189435.266042:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000921cb3a0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654955392:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776189435.266124:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000921cb3a0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654955392:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 103us (321us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189435.266131:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1474 00000100:00100000:1.0:1776189435.267419:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654955456 00000100:00100000:1.0:1776189435.268294:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654955520 00000100:00100000:2.0:1776189435.270012:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9992 00000100:00100000:2.0:1776189435.270048:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0895cec pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654955456:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776189435.270206:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0895cec pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654955456:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 160us (4541us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189435.270214:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9992 00000100:00100000:2.0:1776189435.270252:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57397 00000100:00100000:2.0:1776189435.270256:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002b22e0b2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654955520:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:2.0:1776189435.270278:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189435.270309:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002b22e0b2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654955520:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 53us (4425us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189435.270315:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57397 00010000:02020000:0.0:1776189435.362077:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189435.580440:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000decf8864 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664345856:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189435.580524:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189435.580757:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d01df670 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664345920:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189435.580798:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189435.588695:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664345856 00000100:00100000:2.0:1776189435.588832:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664345920 00000100:00100000:0.0:1776189435.588872:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6832 00000100:00100000:0.0:1776189435.588880:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ace99b9b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664345856:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189435.588917:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189435.589006:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ace99b9b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664345856:12345-0@lo:13:osp-pre-1-0.0 Request processed in 126us (8482us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189435.589014:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6832 00000100:00100000:0.0:1776189435.589047:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6833 00000100:00100000:0.0:1776189435.589052:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d52de5d9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664345920:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776189435.589069:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189435.589097:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d52de5d9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664345920:12345-0@lo:13:osp-pre-0-0.0 Request processed in 44us (8310us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189435.589103:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6833 00000100:00100000:0.0:1776189435.589152:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000decf8864 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664345856:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189435.596525:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d01df670 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664345920:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189435.612295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189435.868297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189435.903475:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189436.124572:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189436.380305:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189436.604505:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000075ad1ba3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664345984:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189436.604580:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189436.604797:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664345984 00000100:00100000:2.0:1776189436.604835:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6834 00000100:00100000:2.0:1776189436.604840:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b34cb071 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664345984:12345-0@lo:13:osp-pre-1-1.0 00000100:00080000:0.0:1776189436.604844:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189436.604849:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189436.604852:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189436.604856:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9578 00000100:00080000:0.0:1776189436.604859:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 11s 00002000:00000024:2.0:1776189436.604862:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00080000:0.0:1776189436.604863:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189436.604878:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000023fe30ce x1862459664346176/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189436.604896:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1776189436.604897:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b34cb071 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664345984:12345-0@lo:13:osp-pre-1-1.0 Request processed in 57us (319us total) trans 0 rc 0/0 00000100:00080000:0.0:1776189436.604899:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189436.604901:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189436.604904:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9578 00000100:00100000:2.0:1776189436.604904:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6834 00000100:00080000:0.0:1776189436.604906:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:0.0:1776189436.604908:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189436.604915:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cbabaed7 x1862459664346304/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189436.604922:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189436.604925:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189436.604926:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189436.604929:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9578 00000100:00080000:0.0:1776189436.604931:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 11s 00000100:00080000:0.0:1776189436.604933:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189436.604943:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000201d0f3c x1862459664346368/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189436.604944:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d9b9c94e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664346112:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189436.604969:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189436.604997:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000be3c8080 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664346560:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189436.605028:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189436.605051:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1776189436.605055:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000014a0e5e7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664346048:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189436.605080:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189436.605093:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000266f5a0c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664346496:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189436.605104:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000023fe30ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664346176:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189436.605112:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189436.605127:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189436.605136:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664346496 00000100:00100000:3.0:1776189436.605137:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cbabaed7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664346304:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189436.605153:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189436.605157:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8583 00000100:00100000:2.0:1776189436.605161:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001aa39bf8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664346496:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189436.605164:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000201d0f3c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664346368:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189436.605203:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189436.605209:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001aa39bf8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664346496:12345-0@lo:400:kworker/u8:0.0 Request processed in 48us (96us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189436.605215:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8583 00000100:00100000:3.0:1776189436.605228:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664346112 00000100:00100000:2.0:1776189436.605230:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664346560 00000100:00100000:2.0:1776189436.605248:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664346176 00000100:00100000:1.0:1776189436.605422:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ccb5377d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664346240:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189436.605448:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189436.621115:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a43a3e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664346624:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189436.621201:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189436.621238:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1776189436.621244:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004d2de401 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664346432:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189436.621271:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189436.621296:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d01df670 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664346688:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189436.621317:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189436.621367:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664346624 00000100:00100000:1.0:1776189436.621413:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664346432 00000100:00100000:1.0:1776189436.621436:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8584 00000100:00100000:1.0:1776189436.621442:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007f770c5d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664346624:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189436.621505:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007f770c5d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664346624:12345-0@lo:400:kworker/u8:0.0 Request processed in 62us (303us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189436.621512:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8584 00000100:00100000:1.0:1776189436.621528:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8585 00000100:00100000:1.0:1776189436.621532:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005c6d79ae pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664346432:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189436.621559:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005c6d79ae pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664346432:12345-0@lo:400:kworker/u8:0.0 Request processed in 26us (288us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189436.621565:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8585 00000100:00100000:2.0:1776189436.622056:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664346304 00000100:00100000:2.0:1776189436.622079:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664346240 00000100:00100000:2.0:1776189436.622107:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9993 00000100:00100000:2.0:1776189436.622113:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d9a1a334 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664346560:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189436.622185:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d9a1a334 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664346560:12345-0@lo:400:kworker/u8:0.0 Request processed in 58us (17143us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189436.622192:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9993 00000100:00100000:2.0:1776189436.622399:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9994 00000100:00100000:2.0:1776189436.622406:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003ff7f95c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664346112:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189436.622609:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003ff7f95c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664346112:12345-0@lo:400:kworker/u8:0.0 Request processed in 202us (17639us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189436.622636:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9994 00000100:00100000:2.0:1776189436.622759:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9995 00000100:00100000:2.0:1776189436.622763:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e7c2ffe2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664346240:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189436.622795:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e7c2ffe2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664346240:12345-0@lo:400:kworker/u8:0.0 Request processed in 31us (17347us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189436.622802:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9995 00000100:00100000:2.0:1776189436.622815:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57398 00000100:00100000:2.0:1776189436.622819:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bc8f2779 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664346176:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189436.622832:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189436.622872:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bc8f2779 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664346176:12345-0@lo:38:kworker/u8:0.0 Request processed in 51us (17745us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189436.622878:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57398 00000100:00100000:2.0:1776189436.622983:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664346688 00000100:00100000:2.0:1776189436.623019:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6835 00000100:00100000:2.0:1776189436.623024:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000058863251 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664346688:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189436.623038:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189436.623069:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000058863251 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664346688:12345-0@lo:13:osp-pre-0-1.0 Request processed in 45us (1751us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189436.623075:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6835 00000100:00100000:2.0:1776189436.623108:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57399 00000100:00100000:2.0:1776189436.623112:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000925fa05a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664346304:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189436.623119:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189436.623142:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000925fa05a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664346304:12345-0@lo:38:kworker/u8:0.0 Request processed in 30us (17989us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189436.623147:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57399 00000100:00100000:3.0:1776189436.623382:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664346048 00000100:00100000:0.0:1776189436.623385:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664346368 00000100:00100000:0.0:1776189436.623420:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9132 00000100:00100000:3.0:1776189436.623423:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1475 00000100:00100000:0.0:1776189436.623428:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000088c2a62e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664346368:12345-0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189436.623431:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008da73e5e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664346048:12345-0@lo:400:kworker/u8:0.0 00010000:02000000:0.0:1776189436.623441:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189436.623575:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000088c2a62e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664346368:12345-0@lo:38:kworker/u8:0.0 Request processed in 147us (18372us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189436.623642:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9132 00000100:00100000:0.0:1776189436.623684:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000014a0e5e7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664346048:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189436.623699:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000266f5a0c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664346496:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189436.623709:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000be3c8080 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664346560:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189436.623720:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d9b9c94e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664346112:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189436.623789:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004d2de401 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664346432:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189436.623959:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000075ad1ba3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664345984:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189436.623971:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d01df670 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664346688:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189436.623990:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ccb5377d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664346240:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189436.624011:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a43a3e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664346624:0@lo:400:kworker/u8:0.0 00000100:00080000:2.0:1776189436.624024:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189436.624030:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189436.624035:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000023fe30ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664346176:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776189436.624048:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189436.624051:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189436.624054:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cbabaed7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664346304:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776189436.624063:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189436.624065:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189436.624067:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000201d0f3c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664346368:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189436.624376:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008da73e5e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664346048:12345-0@lo:400:kworker/u8:0.0 Request processed in 42us (18390us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189436.624384:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1475 00010000:02020000:0.0:1776189436.638873:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189436.892302:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189436.924416:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189437.148292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189437.404238:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189437.660510:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189437.916960:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189437.948392:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189438.174083:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189438.428349:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189438.684303:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189438.940335:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189438.974814:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189439.196290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189439.453786:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189439.708641:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189439.964292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189439.996392:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189440.220291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189440.384533:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189440.384583:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189440.384609:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189440.384694:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654955712 00000100:00100000:1.0:1776189440.384742:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654955648 00000100:00100000:1.0:1776189440.384772:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9996 00000100:00100000:1.0:1776189440.384779:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006047c8f7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654955648:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189440.384858:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006047c8f7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654955648:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 78us (248us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189440.384866:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9996 00000100:00100000:1.0:1776189440.384885:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57400 00000100:00100000:1.0:1776189440.384889:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000075ad1ba3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654955712:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776189440.384915:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189440.384953:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000075ad1ba3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654955712:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 62us (439us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189440.384959:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57400 00000100:00100000:3.0:1776189440.389470:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654955584 00000100:00100000:3.0:1776189440.389594:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1476 00000100:00100000:3.0:1776189440.389600:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c3d39af pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654955584:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189440.389687:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c3d39af pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654955584:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 86us (5103us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189440.389697:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1476 00010000:02020000:0.0:1776189440.476282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189440.700752:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc8ba9ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664346752:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189440.700833:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189440.700954:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664346752 00000100:00100000:2.0:1776189440.701013:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6836 00000100:00100000:2.0:1776189440.701019:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005d3cfd23 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664346752:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189440.701045:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189440.701102:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005d3cfd23 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664346752:12345-0@lo:13:osp-pre-0-0.0 Request processed in 82us (270us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189440.701109:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6836 00000100:00100000:1.0:1776189440.701409:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc8ba9ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664346752:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189440.702465:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776189440.702483:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a03700f8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664346816:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189440.702537:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189440.702620:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664346816 00000100:00100000:3.0:1776189440.702672:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6837 00000100:00100000:3.0:1776189440.702677:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cc8ba9ef pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664346816:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776189440.702700:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189440.702754:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cc8ba9ef pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664346816:12345-0@lo:13:osp-pre-1-0.0 Request processed in 76us (217us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189440.702760:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6837 00000100:00100000:1.0:1776189440.703190:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a03700f8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664346816:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189440.733379:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189440.988255:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189441.021445:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189441.245218:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189441.501520:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189441.724615:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776189441.724637:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8bc9a84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664346880:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189441.724704:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1776189441.724924:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189441.724929:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189441.724932:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189441.724936:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9583 00000100:00080000:2.0:1776189441.724939:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:2.0:1776189441.724944:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189441.724963:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005f3b872c x1862459664347072/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189441.724988:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189441.724990:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189441.724992:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189441.724995:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9583 00000100:00080000:2.0:1776189441.724996:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00080000:2.0:1776189441.724999:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189441.725007:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000087513f3f x1862459664347200/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189441.725015:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189441.725017:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189441.725018:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189441.725021:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9583 00000100:00080000:2.0:1776189441.725023:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00080000:2.0:1776189441.725025:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189441.725035:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000053ddeb10 x1862459664347264/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189441.725076:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005f3b872c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664347072:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189441.725106:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189441.725122:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000087513f3f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664347200:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189441.725141:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189441.725153:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000053ddeb10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664347264:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189441.725196:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189441.725232:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664347072 00000100:00100000:2.0:1776189441.725259:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664347200 00000100:00100000:2.0:1776189441.725274:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57401 00000100:00100000:2.0:1776189441.725279:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000040960a3b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664347072:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189441.725296:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189441.725334:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000040960a3b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664347072:12345-0@lo:38:kworker/u8:0.0 Request processed in 54us (228us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189441.725340:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57401 00000100:00100000:2.0:1776189441.725357:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57402 00000100:00100000:2.0:1776189441.725359:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000945ba48a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664347200:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189441.725366:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189441.725385:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000945ba48a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664347200:12345-0@lo:38:kworker/u8:0.0 Request processed in 25us (244us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189441.725390:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57402 00000100:00100000:2.0:1776189441.725408:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664347264 00000100:00100000:2.0:1776189441.725423:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9133 00000100:00100000:2.0:1776189441.725426:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b7713fdb pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664347264:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189441.725432:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189441.725710:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007c812403 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664347008:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189441.725736:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189441.725752:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e8ddaf27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664347456:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189441.725769:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189441.725782:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1776189441.725786:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000006c1656a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664347136:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189441.725802:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189441.725812:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b696663d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664347520:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189441.725831:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189441.725856:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664347520 00000100:00100000:1.0:1776189441.725883:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8586 00000100:00100000:1.0:1776189441.725887:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d5cfaed8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664347520:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189441.725939:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d5cfaed8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664347520:12345-0@lo:400:kworker/u8:0.0 Request processed in 51us (108us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189441.725945:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8586 00000100:00100000:1.0:1776189441.725965:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664347008 00000100:00100000:1.0:1776189441.725978:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664347456 00000100:00100000:1.0:1776189441.725986:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664347136 00000100:00100000:1.0:1776189441.725996:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9997 00000100:00100000:1.0:1776189441.725999:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a285084b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664347008:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189441.726025:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a285084b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664347008:12345-0@lo:400:kworker/u8:0.0 Request processed in 25us (289us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189441.726030:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9997 00000100:00100000:1.0:1776189441.726040:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9998 00000100:00100000:1.0:1776189441.726044:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000412c3b83 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664347456:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189441.726067:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000412c3b83 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664347456:12345-0@lo:400:kworker/u8:0.0 Request processed in 22us (297us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189441.726072:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9998 00000100:00100000:1.0:1776189441.726082:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9999 00000100:00100000:1.0:1776189441.726085:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e7e6067e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664347136:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189441.726108:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e7e6067e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664347136:12345-0@lo:400:kworker/u8:0.0 Request processed in 22us (305us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189441.726113:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9999 00000100:00100000:1.0:1776189441.726143:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000006c1656a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664347136:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189441.726151:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b696663d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664347520:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189441.726158:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007c812403 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664347008:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189441.726165:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e8ddaf27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664347456:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189441.726283:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000048a53cb2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664347328:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189441.726313:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189441.726330:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002f5aeefd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664347584:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189441.726350:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189441.726364:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1776189441.726369:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002c2b2af4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664346944:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189441.726394:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189441.726410:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005c6b517b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664347392:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189441.726427:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189441.726582:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664346944 00000100:00100000:0.0:1776189441.726606:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1477 00000100:00100000:0.0:1776189441.726609:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e858e44a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664346944:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189441.726638:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e858e44a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664346944:12345-0@lo:400:kworker/u8:0.0 Request processed in 28us (244us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189441.726643:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1477 00000100:00100000:0.0:1776189441.726665:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664347328 00000100:00100000:0.0:1776189441.726686:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664347392 00000100:00100000:0.0:1776189441.726700:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8587 00000100:00100000:0.0:1776189441.726704:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002a43a3e2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664347328:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189441.726730:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002a43a3e2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664347328:12345-0@lo:400:kworker/u8:0.0 Request processed in 25us (417us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189441.726735:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8587 00000100:00100000:0.0:1776189441.726743:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8588 00000100:00100000:0.0:1776189441.726746:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004d2de401 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664347392:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189441.726764:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004d2de401 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664347392:12345-0@lo:400:kworker/u8:0.0 Request processed in 17us (337us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189441.726769:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8588 00000100:00100000:0.0:1776189441.726797:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002c2b2af4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664346944:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189441.726806:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005c6b517b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664347392:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189441.726812:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000048a53cb2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664347328:0@lo:400:kworker/u8:0.0 00000100:00080000:0.0:1776189441.726854:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189441.726857:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189441.726860:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f3b872c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664347072:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776189441.726867:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189441.726870:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189441.726871:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000087513f3f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664347200:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189441.726975:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b7713fdb pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664347264:12345-0@lo:38:kworker/u8:0.0 Request processed in 1546us (1777us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189441.726981:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9133 00000100:00100000:3.0:1776189441.727048:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664346880 00000100:00100000:3.0:1776189441.727077:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664347584 00000100:00100000:3.0:1776189441.727093:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6838 00000100:00100000:3.0:1776189441.727096:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ee179fa1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664346880:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189441.727109:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189441.727142:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ee179fa1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664346880:12345-0@lo:13:osp-pre-1-1.0 Request processed in 45us (2439us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189441.727148:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6838 00000100:00100000:3.0:1776189441.727158:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6839 00000100:00100000:3.0:1776189441.727161:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ccb5377d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664347584:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776189441.727168:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189441.727213:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ccb5377d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664347584:12345-0@lo:13:osp-pre-0-1.0 Request processed in 51us (863us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189441.727218:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6839 00000100:00100000:3.0:1776189441.727256:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8bc9a84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664346880:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189441.727276:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002f5aeefd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664347584:0@lo:13:osp-pre-0-1.0 00000100:00080000:0.0:1776189441.728677:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189441.728686:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189441.728692:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000053ddeb10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664347264:0@lo:38:kworker/u8:0.0 00010000:02020000:2.0:1776189441.756560:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189442.012409:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189442.045115:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189442.268918:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189442.524365:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189442.783218:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189443.036315:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189443.068588:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189443.292286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189443.548977:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189443.804282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189444.060422:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189444.092945:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189444.317431:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189444.572934:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189444.828308:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189445.084441:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189445.118552:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189445.340295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189445.500339:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189445.500388:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189445.500455:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189445.500860:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654955904 00000100:00100000:0.0:1776189445.500941:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654955776 00000100:00100000:0.0:1776189445.500982:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1478 00000100:00100000:0.0:1776189445.500989:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ff8d0a76 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654955776:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189445.501057:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ff8d0a76 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654955776:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 68us (668us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189445.501065:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1478 00000100:00100000:1.0:1776189445.502894:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654955840 00000100:00100000:2.0:1776189445.504620:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10000 00000100:00100000:2.0:1776189445.504632:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ed7f8766 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654955840:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189445.504723:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ed7f8766 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654955840:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 91us (4278us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189445.504731:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10000 00000100:00100000:2.0:1776189445.504760:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57403 00000100:00100000:2.0:1776189445.504764:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000a2a0606 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654955904:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776189445.504780:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189445.504805:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000a2a0606 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654955904:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 40us (4475us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189445.504821:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57403 00010000:02020000:2.0:1776189445.596284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189445.820512:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776189445.820533:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dfad713d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664347648:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189445.820607:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189445.820707:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664347648 00000100:00100000:1.0:1776189445.820762:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6840 00000100:00100000:1.0:1776189445.820768:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e5e4d83b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664347648:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189445.820796:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189445.820861:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e5e4d83b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664347648:12345-0@lo:13:osp-pre-0-0.0 Request processed in 92us (255us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189445.820868:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6840 00000100:00100000:0.0:1776189445.820888:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dfad713d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664347648:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189445.821056:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009fe6d891 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664347712:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189445.821085:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189445.821167:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664347712 00000100:00100000:2.0:1776189445.821223:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6841 00000100:00100000:2.0:1776189445.821227:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005a7b82f1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664347712:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189445.821243:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189445.821282:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005a7b82f1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664347712:12345-0@lo:13:osp-pre-1-0.0 Request processed in 54us (197us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189445.821288:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6841 00000100:00100000:3.0:1776189445.821366:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009fe6d891 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664347712:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189445.852306:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189446.108287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189446.142164:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189446.364659:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189446.620308:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:2.0:1776189446.844424:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189446.844431:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189446.844434:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189446.844442:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9588 00000100:00080000:2.0:1776189446.844447:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776189446.844453:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189446.844457:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e48fdf91 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664347776:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189446.844476:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002477597b x1862459664347904/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189446.844501:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189446.844504:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189446.844506:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189446.844509:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9588 00000100:00080000:2.0:1776189446.844511:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776189446.844514:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189446.844526:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b696663d x1862459664348032/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189446.844535:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1776189446.844538:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189446.844542:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189446.844544:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189446.844548:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9588 00000100:00080000:2.0:1776189446.844550:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776189446.844553:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189446.844566:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c27afb9e x1862459664348096/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189446.844614:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664347776 00000100:00100000:3.0:1776189446.844651:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1479 00000100:00100000:2.0:1776189446.844656:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c995aa10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664347840:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189446.844659:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f07038bd pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664347776:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189446.844743:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002477597b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664347904:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189446.845602:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189446.845679:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000002d6a3fa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664348288:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189446.845708:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189446.845728:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000502b3e4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664348544:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189446.845744:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189446.845753:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1776189446.845757:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003da19f87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664347968:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189446.845769:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189446.845905:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba5f0e8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664348352:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189446.845927:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189446.846342:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f07038bd pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664347776:12345-0@lo:400:kworker/u8:1.0 Request processed in 1683us (1809us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189446.846352:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1479 00000100:00100000:3.0:1776189446.846394:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009fe6d891 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664348224:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189446.846420:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189446.846436:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f8dcf939 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664348480:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189446.846458:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189446.846485:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e48fdf91 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664347776:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189446.846504:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1776189446.846509:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008206d902 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664348160:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189446.846531:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189446.846547:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000838deece pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664348416:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189446.846567:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189446.846595:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664348288 00000100:00100000:3.0:1776189446.846625:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664348480 00000100:00100000:3.0:1776189446.846639:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664348160 00000100:00100000:3.0:1776189446.846654:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664348416 00000100:00100000:3.0:1776189446.846684:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8589 00000100:00100000:3.0:1776189446.846688:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e0c8fd67 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664348288:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189446.846726:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e0c8fd67 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664348288:12345-0@lo:400:kworker/u8:1.0 Request processed in 37us (1017us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189446.846731:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664347840 00000100:00100000:3.0:1776189446.846732:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8589 00000100:00100000:3.0:1776189446.846746:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8590 00000100:00100000:3.0:1776189446.846751:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000076de9aaf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664348480:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189446.846771:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664347968 00000100:00100000:1.0:1776189446.846814:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10001 00000100:00100000:1.0:1776189446.846818:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fbad805c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664347840:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189446.848094:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000076de9aaf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664348480:12345-0@lo:400:kworker/u8:1.0 Request processed in 1340us (1633us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189446.848103:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8590 00000100:00100000:1.0:1776189446.848130:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fbad805c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664347840:12345-0@lo:400:kworker/u8:1.0 Request processed in 1308us (2526us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189446.848139:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10001 00000100:00100000:1.0:1776189446.848169:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664348352 00000100:00100000:1.0:1776189446.848219:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10002 00000100:00100000:1.0:1776189446.848225:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009f7be854 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664347968:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189446.848357:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8591 00000100:00100000:3.0:1776189446.848363:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000540d168e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664348160:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189446.848400:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000540d168e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664348160:12345-0@lo:400:kworker/u8:1.0 Request processed in 37us (1868us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189446.848405:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8591 00000100:00100000:3.0:1776189446.848415:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8592 00000100:00100000:3.0:1776189446.848419:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a109d002 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664348416:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189446.848439:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a109d002 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664348416:12345-0@lo:400:kworker/u8:1.0 Request processed in 20us (1873us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189446.848444:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8592 00000100:00100000:3.0:1776189446.848490:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008206d902 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664348160:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189446.848498:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000838deece pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664348416:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189446.848508:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f8dcf939 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664348480:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189446.848553:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664348544 00000100:00100000:1.0:1776189446.848561:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009f7be854 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664347968:12345-0@lo:400:kworker/u8:1.0 Request processed in 335us (2789us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189446.848568:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10002 00000100:00100000:3.0:1776189446.848574:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664348224 00000100:00100000:1.0:1776189446.848587:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10003 00000100:00100000:1.0:1776189446.848592:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000305a0bea pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664348352:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189446.848597:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6842 00000100:00100000:3.0:1776189446.848602:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f40573b9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664348544:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776189446.848618:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189446.848622:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000305a0bea pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664348352:12345-0@lo:400:kworker/u8:1.0 Request processed in 29us (2694us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189446.848629:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10003 00000100:00100000:3.0:1776189446.848655:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f40573b9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664348544:12345-0@lo:13:osp-pre-0-1.0 Request processed in 51us (2908us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189446.848672:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6842 00000100:00100000:3.0:1776189446.848687:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6843 00000100:00100000:3.0:1776189446.848691:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000db0032a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664348224:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189446.848701:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189446.848737:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000db0032a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664348224:12345-0@lo:13:osp-pre-1-1.0 Request processed in 45us (2317us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189446.848743:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6843 00000100:00100000:3.0:1776189446.848777:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009fe6d891 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664348224:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189446.848894:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba5f0e8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664348352:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189446.848910:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000502b3e4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664348544:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189446.848920:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c995aa10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664347840:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189446.848929:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000002d6a3fa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664348288:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189446.848956:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003da19f87 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664347968:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189446.851108:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189446.851156:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b696663d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664348032:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189446.851200:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189446.851230:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c27afb9e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664348096:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189446.851255:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189446.851481:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664347904 00000100:00100000:1.0:1776189446.851517:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664348032 00000100:00100000:1.0:1776189446.851534:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57404 00000100:00100000:1.0:1776189446.851539:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d9b9c94e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664347904:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189446.851572:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189446.851618:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d9b9c94e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664347904:12345-0@lo:38:kworker/u8:1.0 Request processed in 78us (517us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189446.851625:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57404 00000100:00100000:1.0:1776189446.851644:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57405 00000100:00100000:1.0:1776189446.851648:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000be3c8080 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664348032:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189446.851655:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189446.851687:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000be3c8080 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664348032:12345-0@lo:38:kworker/u8:1.0 Request processed in 38us (487us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189446.851692:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57405 00000100:00100000:3.0:1776189446.852169:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664348096 00000100:00100000:3.0:1776189446.852226:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9134 00000100:00100000:3.0:1776189446.852229:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000266f5a0c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664348096:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189446.852237:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189446.852269:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000266f5a0c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664348096:12345-0@lo:38:kworker/u8:1.0 Request processed in 38us (1014us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189446.852274:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9134 00000100:00080000:0.0:1776189446.852574:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189446.852579:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189446.852583:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002477597b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664347904:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776189446.852593:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189446.852596:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189446.852598:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b696663d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664348032:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776189446.852606:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189446.852608:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189446.852610:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c27afb9e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664348096:0@lo:38:kworker/u8:1.0 00010000:02020000:2.0:1776189446.877513:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189447.132965:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189447.164417:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189447.388289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189447.644942:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189447.900328:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189448.156299:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189448.190425:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189448.412894:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189448.669097:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189448.924813:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189449.180349:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189449.212501:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189449.436284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189449.694305:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189449.950924:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189450.204282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189450.238301:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189450.460296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189450.618137:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189450.618230:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189450.618401:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654956032 00000100:00100000:1.0:1776189450.618458:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10004 00000100:00100000:0.0:1776189450.619335:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654955968 00000100:00100000:0.0:1776189450.619385:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1480 00000100:00100000:0.0:1776189450.619392:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005039a00d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654955968:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189450.619740:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005039a00d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654955968:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 73us (1337us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189450.619750:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1480 00000100:00100000:1.0:1776189450.620249:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000002d6a3fa pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654956032:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189450.620308:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000002d6a3fa pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654956032:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 62us (2078us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189450.623358:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10004 00000100:00100000:2.0:1776189450.624671:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189450.625050:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654956096 00000100:00100000:1.0:1776189450.625092:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57406 00000100:00100000:1.0:1776189450.625097:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba5f0e8e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654956096:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776189450.625116:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189450.625253:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba5f0e8e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654956096:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 154us (587us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189450.625260:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57406 00010000:02020000:0.0:1776189450.716232:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189450.941006:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a943cb24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664348608:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189450.941078:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189450.941156:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664348608 00000100:00100000:2.0:1776189450.941236:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6844 00000100:00100000:2.0:1776189450.941243:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000014a0e5e7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664348608:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189450.941272:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189450.941903:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009fe6d891 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664348672:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189450.941948:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189450.941983:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664348672 00000100:00100000:1.0:1776189450.942009:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6845 00000100:00100000:1.0:1776189450.942012:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004447f539 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664348672:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189450.942030:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189450.942063:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004447f539 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664348672:12345-0@lo:13:osp-pre-1-0.0 Request processed in 49us (115us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189450.942070:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6845 00000100:00100000:1.0:1776189450.942115:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009fe6d891 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664348672:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189450.942665:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000014a0e5e7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664348608:12345-0@lo:13:osp-pre-0-0.0 Request processed in 1420us (1586us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189450.942674:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6844 00000100:00100000:0.0:1776189450.942743:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a943cb24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664348608:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189450.972328:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189451.228965:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189451.260683:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189451.484383:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189451.740532:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189451.964708:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c6bc9cdf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664348736:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189451.964775:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189451.964871:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664348736 00000100:00100000:1.0:1776189451.964918:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6846 00000100:00100000:1.0:1776189451.964923:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000099b8eb0d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664348736:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189451.964944:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189451.964987:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000099b8eb0d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664348736:12345-0@lo:13:osp-pre-0-1.0 Request processed in 63us (213us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189451.964994:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6846 00000100:00100000:0.0:1776189451.965324:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c6bc9cdf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664348736:0@lo:13:osp-pre-0-1.0 00000100:00080000:3.0:1776189451.965568:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189451.965573:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189451.965577:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189451.965583:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9593 00000100:00080000:3.0:1776189451.965586:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189451.965589:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189451.965609:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000064bc62e8 x1862459664348928/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189451.965638:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189451.965642:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189451.965644:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189451.965647:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9593 00000100:00080000:3.0:1776189451.965649:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189451.965652:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189451.965661:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000dd5e571c x1862459664349056/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189451.965678:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189451.965681:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189451.965682:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189451.965686:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9593 00000100:00080000:3.0:1776189451.965687:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189451.965690:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189451.965700:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005f37d44e x1862459664349120/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189451.965743:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008950b44b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664348992:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189451.965778:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189451.965797:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009fe6d891 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664349376:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189451.965819:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189451.965835:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776189451.965839:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008ceb4933 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664349184:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189451.965860:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189451.965893:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664348992 00000100:00100000:1.0:1776189451.966150:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ab2fefce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664348800:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189451.966221:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189451.966236:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004dd53b72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664349248:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189451.966255:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189451.966271:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1776189451.966276:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d0c1478c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664348864:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189451.966296:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189451.966320:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cf6b5d65 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664349312:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189451.966338:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189451.966364:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664349376 00000100:00100000:1.0:1776189451.966388:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664349184 00000100:00100000:1.0:1776189451.966402:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664349248 00000100:00100000:1.0:1776189451.966417:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8593 00000100:00100000:1.0:1776189451.966421:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000decf8864 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664349376:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189451.966465:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000decf8864 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664349376:12345-0@lo:400:kworker/u8:0.0 Request processed in 43us (645us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189451.966472:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8593 00000100:00100000:1.0:1776189451.966484:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8594 00000100:00100000:1.0:1776189451.966487:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d45cc7cd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664349184:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189451.966511:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d45cc7cd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664349184:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (652us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189451.966517:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8594 00000100:00100000:1.0:1776189451.966526:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8595 00000100:00100000:1.0:1776189451.966529:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005cdf5f34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664349248:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189451.966555:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005cdf5f34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664349248:12345-0@lo:400:kworker/u8:0.0 Request processed in 25us (300us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189451.966561:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8595 00000100:00100000:1.0:1776189451.966585:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664348864 00000100:00100000:1.0:1776189451.966605:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664349312 00000100:00100000:1.0:1776189451.966620:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10005 00000100:00100000:1.0:1776189451.966624:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a03700f8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664348992:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189451.966651:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a03700f8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664348992:12345-0@lo:400:kworker/u8:0.0 Request processed in 26us (873us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189451.966657:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10005 00000100:00100000:1.0:1776189451.966681:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10006 00000100:00100000:1.0:1776189451.966685:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005cd76944 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664348864:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189451.966708:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005cd76944 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664348864:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (412us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189451.966713:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10006 00000100:00100000:1.0:1776189451.966724:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10007 00000100:00100000:1.0:1776189451.966727:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009b864aa6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664349312:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189451.966748:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009b864aa6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664349312:12345-0@lo:400:kworker/u8:0.0 Request processed in 20us (410us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189451.966754:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10007 00000100:00100000:1.0:1776189451.966800:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d0c1478c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664348864:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189451.966811:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cf6b5d65 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664349312:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189451.966818:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004dd53b72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664349248:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189451.966863:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008ceb4933 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664349184:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189451.966873:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008950b44b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664348992:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189451.966881:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009fe6d891 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664349376:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189451.967457:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064bc62e8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664348928:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189451.967494:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189451.967513:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dd5e571c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664349056:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189451.967534:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189451.967549:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005f37d44e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664349120:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189451.967572:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189451.967604:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664348800 00000100:00100000:0.0:1776189451.967634:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1481 00000100:00100000:0.0:1776189451.967638:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000080a5522a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664348800:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189451.967692:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000080a5522a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664348800:12345-0@lo:400:kworker/u8:0.0 Request processed in 52us (1471us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189451.967698:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1481 00000100:00100000:0.0:1776189451.967727:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664349120 00000100:00100000:0.0:1776189451.967748:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9135 00000100:00100000:0.0:1776189451.967752:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004920d098 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664349120:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189451.967765:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189451.967796:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004920d098 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664349120:12345-0@lo:38:kworker/u8:0.0 Request processed in 43us (224us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189451.967802:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9135 00000100:00100000:0.0:1776189451.967823:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664348928 00000100:00100000:0.0:1776189451.967841:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664349056 00000100:00100000:0.0:1776189451.967856:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57407 00000100:00100000:0.0:1776189451.967860:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000056e9ad0b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664348928:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189451.967867:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189451.967890:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000056e9ad0b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664348928:12345-0@lo:38:kworker/u8:0.0 Request processed in 29us (397us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189451.967896:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57407 00000100:00100000:0.0:1776189451.967906:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57408 00000100:00100000:0.0:1776189451.967909:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053c82750 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664349056:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189451.967916:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189451.967937:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053c82750 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664349056:12345-0@lo:38:kworker/u8:0.0 Request processed in 26us (402us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189451.967942:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57408 00000100:00080000:0.0:1776189451.967975:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189451.967979:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189451.967983:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064bc62e8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664348928:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776189451.967990:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189451.967992:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189451.967994:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dd5e571c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664349056:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776189451.968001:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189451.968003:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189451.968006:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f37d44e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664349120:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189451.968033:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019f375fb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664349440:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189451.968060:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189451.968089:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664349440 00000100:00100000:0.0:1776189451.968111:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6847 00000100:00100000:0.0:1776189451.968114:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000403a47d8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664349440:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189451.968126:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189451.968155:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000403a47d8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664349440:12345-0@lo:13:osp-pre-1-1.0 Request processed in 40us (95us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189451.968160:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6847 00000100:00100000:1.0:1776189451.968616:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019f375fb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664349440:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189451.968647:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ab2fefce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664348800:0@lo:400:kworker/u8:0.0 00010000:02020000:0.0:1776189451.996323:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189452.252289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189452.284621:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189452.510107:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189452.764321:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189453.020362:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189453.276320:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189453.308369:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189453.532297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189453.788367:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189454.047032:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189454.300443:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189454.332697:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189454.556285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189454.812358:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189455.068290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189455.324515:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189455.356648:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189455.580457:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189455.740664:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189455.740759:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654956288 00000100:00100000:1.0:1776189455.740809:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57409 00000100:00100000:1.0:1776189455.740815:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002a43a3e2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654956288:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776189455.740836:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189455.740911:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002a43a3e2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654956288:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 95us (253us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189455.740918:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57409 00000100:00100000:3.0:1776189455.752413:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189455.752516:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189455.752803:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654956160 00000100:00100000:0.0:1776189455.752877:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1482 00000100:00100000:0.0:1776189455.752883:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f99b7a85 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654956160:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189455.752972:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f99b7a85 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654956160:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 88us (457us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189455.752980:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1482 00000100:00100000:2.0:1776189455.757576:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654956224 00000100:00100000:2.0:1776189455.757663:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10008 00000100:00100000:2.0:1776189455.757670:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005ad21a28 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654956224:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189455.757768:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005ad21a28 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654956224:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 97us (5364us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189455.757776:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10008 00010000:02020000:0.0:1776189455.836468:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189456.060544:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ab2fefce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664349504:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189456.060620:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189456.060828:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664349504 00000100:00100000:2.0:1776189456.060882:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6848 00000100:00100000:2.0:1776189456.060889:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c6bc9cdf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664349504:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189456.060919:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189456.060959:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c6bc9cdf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664349504:12345-0@lo:13:osp-pre-1-0.0 Request processed in 70us (340us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189456.060967:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6848 00000100:00100000:1.0:1776189456.061060:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ab2fefce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664349504:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189456.062030:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a943cb24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664349568:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189456.062073:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189456.062203:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664349568 00000100:00100000:3.0:1776189456.062253:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6849 00000100:00100000:3.0:1776189456.062257:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004d2de401 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664349568:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189456.062278:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189456.062893:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004d2de401 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664349568:12345-0@lo:13:osp-pre-0-0.0 Request processed in 633us (819us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189456.062904:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6849 00000100:00100000:0.0:1776189456.062976:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a943cb24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664349568:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189456.092307:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189456.348623:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189456.380861:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189456.605159:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189456.863429:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:2.0:1776189457.084548:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189457.084556:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189457.084570:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189457.084578:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9598 00000100:00080000:2.0:1776189457.084583:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776189457.084588:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189457.084596:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038a7e045 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664349632:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189457.084609:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003cd25a35 x1862459664349760/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189457.084646:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189457.084650:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189457.084651:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189457.084654:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9598 00000100:00080000:2.0:1776189457.084656:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776189457.084659:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189457.084670:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189457.084672:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ed29900a x1862459664349888/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189457.084681:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189457.084684:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189457.084685:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189457.084688:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9598 00000100:00080000:2.0:1776189457.084690:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776189457.084692:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189457.084702:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006302d616 x1862459664349952/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189457.084706:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776189457.084716:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000032d9de2c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664349696:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189457.084749:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189457.084769:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003cd25a35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664349760:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189457.084774:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052e42ae8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664350080:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189457.084809:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189457.084814:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189457.084830:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed29900a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664349888:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189457.084848:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776189457.084856:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c68cc49e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664350144:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189457.084861:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189457.084879:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006302d616 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664349952:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189457.084887:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189457.084911:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189457.084922:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664349632 00000100:00100000:2.0:1776189457.084945:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a45b748c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664350016:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189457.084963:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1483 00000100:00100000:3.0:1776189457.084970:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000afb92d61 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664349632:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189457.084977:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189457.084995:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000068da6036 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664350272:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189457.085027:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189457.085037:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000afb92d61 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664349632:12345-0@lo:400:kworker/u8:0.0 Request processed in 65us (369us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189457.085046:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1483 00000100:00100000:2.0:1776189457.085047:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:2.0:1776189457.085054:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a4221496 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664349824:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189457.085081:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189457.085084:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664349952 00000100:00100000:2.0:1776189457.085099:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3d6c201 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664350208:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189457.085106:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9136 00000100:00100000:3.0:1776189457.085111:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c27afb9e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664349952:12345-0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189457.085126:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:02000000:3.0:1776189457.085127:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189457.085157:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664350080 00000100:00100000:3.0:1776189457.085167:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c27afb9e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664349952:12345-0@lo:38:kworker/u8:0.0 Request processed in 54us (256us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189457.085201:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9136 00000100:00100000:2.0:1776189457.085221:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664350016 00000100:00100000:3.0:1776189457.085244:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038a7e045 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664349632:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189457.085246:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664350272 00000100:00100000:1.0:1776189457.085262:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664349696 00000100:00100000:2.0:1776189457.085272:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664350208 00000100:00100000:1.0:1776189457.085300:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664349760 00000100:00100000:1.0:1776189457.085313:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664349888 00000100:00100000:1.0:1776189457.085320:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664350144 00000100:00100000:1.0:1776189457.085331:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664349824 00000100:00100000:1.0:1776189457.085344:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10009 00000100:00100000:1.0:1776189457.085349:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f37d44e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664349696:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189457.085387:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10010 00000100:00100000:0.0:1776189457.085393:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000064bc62e8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664350144:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189457.085398:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f37d44e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664349696:12345-0@lo:400:kworker/u8:0.0 Request processed in 46us (647us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189457.085406:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10009 00000100:00100000:1.0:1776189457.085422:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10011 00000100:00100000:1.0:1776189457.085427:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008206d902 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664349824:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189457.085440:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000064bc62e8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664350144:12345-0@lo:400:kworker/u8:0.0 Request processed in 45us (553us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189457.085447:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10010 00000100:00100000:0.0:1776189457.085470:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57410 00000100:00100000:1.0:1776189457.085471:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008206d902 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664349824:12345-0@lo:400:kworker/u8:0.0 Request processed in 40us (387us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189457.085478:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000efff0bc3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664349760:12345-0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189457.085483:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10011 00010000:02000000:0.0:1776189457.085489:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189457.085500:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57411 00000100:00100000:1.0:1776189457.085504:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000019f375fb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664349888:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189457.085514:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189457.085519:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000efff0bc3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664349760:12345-0@lo:38:kworker/u8:0.0 Request processed in 42us (710us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189457.085527:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57410 00000100:00100000:1.0:1776189457.085547:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000019f375fb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664349888:12345-0@lo:38:kworker/u8:0.0 Request processed in 40us (689us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189457.085554:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57411 00000100:00100000:1.0:1776189457.085607:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c68cc49e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664350144:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189457.085610:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776189457.085621:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000032d9de2c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664349696:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189457.085622:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000201d0f3c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664350336:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189457.085657:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189457.085690:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8596 00000100:00100000:2.0:1776189457.085695:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd5e571c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664350080:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189457.085730:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd5e571c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664350080:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (919us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189457.085736:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8596 00000100:00100000:2.0:1776189457.085749:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8597 00000100:00100000:2.0:1776189457.085752:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f8dcf939 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664350016:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189457.085774:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f8dcf939 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664350016:12345-0@lo:400:kworker/u8:0.0 Request processed in 21us (802us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189457.085779:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8597 00000100:00100000:2.0:1776189457.085788:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8598 00000100:00100000:2.0:1776189457.085791:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000838deece pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664350272:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189457.085812:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000838deece pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664350272:12345-0@lo:400:kworker/u8:0.0 Request processed in 20us (786us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189457.085818:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8598 00000100:00100000:2.0:1776189457.085827:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8599 00000100:00100000:2.0:1776189457.085830:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e532ce7c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664350208:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189457.085852:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e532ce7c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664350208:12345-0@lo:400:kworker/u8:0.0 Request processed in 21us (728us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189457.085857:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8599 00000100:00100000:3.0:1776189457.086829:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664350336 00000100:00080000:1.0:1776189457.087134:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189457.087141:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189457.087147:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003cd25a35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664349760:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776189457.087163:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189457.087165:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189457.087167:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed29900a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664349888:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776189457.087191:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189457.087194:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189457.087197:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006302d616 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664349952:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189457.087266:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000502b3e4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664350400:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189457.087307:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189457.087340:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664350400 00000100:00100000:1.0:1776189457.087376:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6850 00000100:00100000:1.0:1776189457.087381:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d4382e95 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664350400:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189457.087398:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189457.087433:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d4382e95 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664350400:12345-0@lo:13:osp-pre-1-1.0 Request processed in 51us (126us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189457.087439:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6850 00000100:00100000:1.0:1776189457.087471:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000502b3e4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664350400:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189457.087746:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a4221496 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664349824:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189457.087762:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3d6c201 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664350208:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189457.087773:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a45b748c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664350016:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189457.087782:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000068da6036 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664350272:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189457.087924:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052e42ae8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664350080:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189457.089232:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6851 00000100:00100000:3.0:1776189457.089238:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cbabaed7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664350336:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776189457.089261:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189457.089336:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cbabaed7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664350336:12345-0@lo:13:osp-pre-0-1.0 Request processed in 97us (3679us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189457.089342:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6851 00000100:00100000:3.0:1776189457.089396:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000201d0f3c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664350336:0@lo:13:osp-pre-0-1.0 00010000:02020000:0.0:1776189457.116313:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189457.373314:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189457.404407:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189457.628289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189457.884286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189458.144672:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189458.396942:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189458.428891:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189458.652475:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189458.908341:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189459.164318:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189459.420292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189459.452359:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189459.676284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189459.932307:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189460.188284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189460.444382:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189460.478416:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189460.700292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189460.863904:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189460.863957:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189460.864234:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654956352 00000100:00100000:0.0:1776189460.864290:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1484 00000100:00100000:0.0:1776189460.864297:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008f85e8f7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654956352:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1776189460.864383:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008f85e8f7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654956352:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 85us (487us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189460.864390:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1484 00000100:00100000:3.0:1776189460.864895:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189460.880270:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654956416 00000100:00100000:2.0:1776189460.880436:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654956480 00000100:00100000:2.0:1776189460.880472:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10012 00000100:00100000:2.0:1776189460.880489:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009fe6d891 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654956416:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1776189460.880498:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57412 00000100:00100000:3.0:1776189460.880505:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008950b44b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654956480:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:3.0:1776189460.880524:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776189460.880596:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008950b44b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654956480:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 90us (15703us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189460.880603:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57412 00000100:00100000:2.0:1776189460.880625:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009fe6d891 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654956416:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 136us (16667us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189460.880634:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10012 00010000:02020000:0.0:1776189460.956461:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189461.180499:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776189461.180828:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008ceb4933 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664350528:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189461.180889:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189461.182584:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000502b3e4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664350464:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189461.182632:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189461.183734:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664350528 00000100:00100000:2.0:1776189461.183825:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664350464 00000100:00100000:1.0:1776189461.184005:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6852 00000100:00100000:1.0:1776189461.184012:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000023fe30ce pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664350528:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189461.184045:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189461.184111:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000023fe30ce pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664350528:12345-0@lo:13:osp-pre-1-0.0 Request processed in 98us (3222us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189461.184118:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6852 00000100:00100000:1.0:1776189461.184136:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6853 00000100:00100000:1.0:1776189461.184140:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000201d0f3c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664350464:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189461.184156:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189461.184199:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000201d0f3c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664350464:12345-0@lo:13:osp-pre-0-0.0 Request processed in 58us (1567us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189461.184204:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6853 00000100:00100000:0.0:1776189461.187574:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008ceb4933 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664350528:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189461.187599:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000502b3e4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664350464:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189461.212536:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189461.468270:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189461.500363:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189461.724269:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189461.981440:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189462.204709:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc4b762b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664350592:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189462.204785:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189462.204847:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664350592 00000100:00100000:2.0:1776189462.204893:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6854 00000100:00100000:2.0:1776189462.204898:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c04acb9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664350592:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189462.204922:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189462.204961:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c04acb9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664350592:12345-0@lo:13:osp-pre-1-1.0 Request processed in 62us (177us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189462.204968:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6854 00000100:00100000:2.0:1776189462.205015:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc4b762b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664350592:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189462.205090:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776189462.205099:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004dd53b72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664350656:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189462.205123:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189462.205243:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664350656 00000100:00100000:1.0:1776189462.205288:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6855 00000100:00100000:1.0:1776189462.205292:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e0c8fd67 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664350656:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189462.205308:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189462.205596:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e0c8fd67 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664350656:12345-0@lo:13:osp-pre-0-1.0 Request processed in 302us (472us total) trans 0 rc 0/0 00000100:00080000:0.0:1776189462.205831:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189462.205836:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189462.205839:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189462.205845:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9604 00000100:00080000:0.0:1776189462.205848:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00080000:0.0:1776189462.205851:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189462.205870:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000862ba2d5 x1862459664350848/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189462.205895:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189462.205898:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189462.205899:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189462.205902:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9604 00000100:00080000:0.0:1776189462.205904:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00080000:0.0:1776189462.205907:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189462.205917:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d6ea221c x1862459664350976/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189462.205924:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189462.205927:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189462.205928:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189462.205930:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9604 00000100:00080000:0.0:1776189462.205932:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 6s 00000100:00080000:0.0:1776189462.205934:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189462.205943:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d2d0e609 x1862459664351040/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189462.205980:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d4da5ec0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664350784:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189462.206011:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189462.206029:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b181f293 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664351232:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189462.206048:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189462.206064:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1776189462.206068:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa8befb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664350720:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189462.206094:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189462.206108:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000476e266d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664351168:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189462.206131:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189462.206159:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664350784 00000100:00100000:0.0:1776189462.206211:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664351232 00000100:00100000:0.0:1776189462.206228:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10013 00000100:00100000:0.0:1776189462.206232:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000037fc1275 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664350784:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189462.206267:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000037fc1275 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664350784:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (256us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189462.206287:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10013 00000100:00100000:0.0:1776189462.206300:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10014 00000100:00100000:0.0:1776189462.206304:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005ecd069c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664351232:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189462.206341:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005ecd069c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664351232:12345-0@lo:400:kworker/u8:0.0 Request processed in 25us (281us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189462.206346:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10014 00000100:00100000:0.0:1776189462.206366:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664351168 00000100:00100000:0.0:1776189462.206393:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8600 00000100:00100000:0.0:1776189462.206396:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cab21754 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664351168:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189462.206425:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cab21754 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664351168:12345-0@lo:400:kworker/u8:0.0 Request processed in 27us (293us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189462.206430:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8600 00000100:00100000:3.0:1776189462.207207:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028933edf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664351104:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189462.207249:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189462.207365:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776189462.207371:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000797ba66f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664350912:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189462.207383:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000476e266d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664351168:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189462.207397:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189462.207399:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d4da5ec0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664350784:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189462.207415:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b181f293 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664351232:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189462.207431:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664350912 00000100:00100000:0.0:1776189462.207463:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664351104 00000100:00100000:3.0:1776189462.207481:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10015 00000100:00100000:3.0:1776189462.207487:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d0c1478c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664350912:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189462.207496:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8601 00000100:00100000:0.0:1776189462.207502:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cf6b5d65 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664351104:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189462.207529:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d0c1478c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664350912:12345-0@lo:400:kworker/u8:0.0 Request processed in 42us (133us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189462.207536:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10015 00000100:00100000:0.0:1776189462.207546:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cf6b5d65 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664351104:12345-0@lo:400:kworker/u8:0.0 Request processed in 42us (296us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189462.207554:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8601 00000100:00100000:3.0:1776189462.207567:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000797ba66f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664350912:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189462.207590:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028933edf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664351104:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189462.207612:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664350720 00000100:00100000:3.0:1776189462.207637:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1485 00000100:00100000:3.0:1776189462.207641:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000034f05964 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664350720:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189462.207681:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000862ba2d5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664350848:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189462.207707:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189462.207723:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d6ea221c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664350976:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189462.207741:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189462.207755:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d2d0e609 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664351040:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189462.207774:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189462.207800:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664350848 00000100:00100000:0.0:1776189462.207816:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664350976 00000100:00100000:0.0:1776189462.207824:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57413 00000100:00100000:0.0:1776189462.207827:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b181f293 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664350848:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189462.207841:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189462.207870:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b181f293 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664350848:12345-0@lo:38:kworker/u8:0.0 Request processed in 42us (164us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189462.207877:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57413 00000100:00100000:0.0:1776189462.207889:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57414 00000100:00100000:0.0:1776189462.207894:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000476e266d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664350976:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189462.207901:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189462.207935:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000476e266d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664350976:12345-0@lo:38:kworker/u8:0.0 Request processed in 41us (193us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189462.207941:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57414 00000100:00100000:0.0:1776189462.207959:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664351040 00000100:00100000:0.0:1776189462.207975:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9137 00000100:00100000:0.0:1776189462.207979:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000073262ad2 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664351040:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189462.207986:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189462.208008:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000073262ad2 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664351040:12345-0@lo:38:kworker/u8:0.0 Request processed in 29us (234us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189462.208013:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9137 00000100:00080000:0.0:1776189462.208043:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189462.208046:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189462.208050:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000862ba2d5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664350848:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776189462.208057:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:3.0:1776189462.208059:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000034f05964 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664350720:12345-0@lo:400:kworker/u8:0.0 Request processed in 408us (1954us total) trans 0 rc 0/0 00000100:00080000:0.0:1776189462.208061:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189462.208067:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d6ea221c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664350976:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189462.208072:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1485 00000100:00080000:0.0:1776189462.208077:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189462.208081:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189462.208085:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d2d0e609 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664351040:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189462.208132:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa8befb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664350720:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189462.208620:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6855 00000100:00100000:2.0:1776189462.208747:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004dd53b72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664350656:0@lo:13:osp-pre-0-1.0 00010000:02020000:0.0:1776189462.236272:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189462.493158:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189462.524465:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189462.749631:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189463.004465:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189463.260917:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189463.516333:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189463.548377:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189463.772327:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189464.028905:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189464.285412:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189464.540320:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189464.572413:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189464.796528:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189465.052281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189465.308781:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189465.564305:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189465.596991:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189465.820522:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189465.979721:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189465.979770:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189465.980437:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654956608 00000100:00100000:2.0:1776189465.980508:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10016 00000100:00100000:2.0:1776189465.980516:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d2d0e609 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654956608:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189465.980591:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d2d0e609 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654956608:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 74us (821us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189465.980598:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10016 00000100:00100000:3.0:1776189465.980752:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654956544 00000100:00100000:3.0:1776189465.980793:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1486 00000100:00100000:3.0:1776189465.980807:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d6b85e7e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654956544:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189465.980852:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d6b85e7e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654956544:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 45us (1140us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189465.980859:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1486 00000100:00100000:0.0:1776189465.982515:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189465.982837:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654956672 00000100:00100000:1.0:1776189465.982884:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57415 00000100:00100000:1.0:1776189465.982891:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d6ea221c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654956672:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776189465.982909:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189465.982972:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d6ea221c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654956672:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 81us (463us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189465.982980:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57415 00010000:02020000:2.0:1776189466.076590:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189466.300689:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007f770c5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664351296:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189466.300762:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189466.300872:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664351296 00000100:00100000:1.0:1776189466.300923:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6856 00000100:00100000:1.0:1776189466.300929:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000052e42ae8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664351296:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189466.300959:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189466.300999:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000052e42ae8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664351296:12345-0@lo:13:osp-pre-0-0.0 Request processed in 70us (239us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189466.301006:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6856 00000100:00100000:2.0:1776189466.301093:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007f770c5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664351296:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189466.301315:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776189466.301325:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004dd53b72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664351360:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189466.301351:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189466.301522:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664351360 00000100:00100000:3.0:1776189466.301556:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6857 00000100:00100000:3.0:1776189466.301561:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000068da6036 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664351360:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776189466.301579:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189466.301620:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000068da6036 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664351360:12345-0@lo:13:osp-pre-1-0.0 Request processed in 58us (268us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189466.301626:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6857 00000100:00100000:2.0:1776189466.301872:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004dd53b72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664351360:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189466.332685:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189466.588287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189466.620505:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189466.844309:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189467.100286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189467.324504:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008ceb4933 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664351424:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189467.324574:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776189467.324599:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189467.324604:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189467.324607:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1776189467.324608:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00080000:1.0:1776189467.324612:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9609 00000100:00100000:0.0:1776189467.324613:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005c6d79ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664351488:0@lo:13:osp-pre-1-1.0 00000100:00080000:1.0:1776189467.324615:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776189467.324620:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189467.324632:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189467.324637:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002f5aeefd x1862459664351680/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189467.324660:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002f5aeefd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664351680:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776189467.324664:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189467.324667:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189467.324668:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189467.324671:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9609 00000100:00080000:1.0:1776189467.324673:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776189467.324677:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189467.324683:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189467.324686:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005be3c9f3 x1862459664351808/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189467.324693:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189467.324696:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1776189467.324697:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005be3c9f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664351808:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776189467.324697:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189467.324700:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9609 00000100:00080000:1.0:1776189467.324702:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776189467.324705:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189467.324715:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189467.324715:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007f770c5d x1862459664351872/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189467.324729:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007f770c5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664351872:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189467.324749:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189467.324758:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664351680 00000100:00100000:0.0:1776189467.324779:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dfad713d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664351616:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189467.324784:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664351808 00000100:00100000:1.0:1776189467.324798:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57416 00000100:00100000:0.0:1776189467.324801:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189467.324803:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001f4e478a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664351680:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189467.324814:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a943cb24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664352064:0@lo:400:kworker/u8:1.0 00010000:02000000:1.0:1776189467.324818:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189467.324831:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189467.324844:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1776189467.324849:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d5cfaed8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664351552:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189467.324853:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001f4e478a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664351680:12345-0@lo:38:kworker/u8:1.0 Request processed in 49us (170us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189467.324859:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57416 00000100:00100000:0.0:1776189467.324871:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189467.324873:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664351616 00000100:00100000:0.0:1776189467.324884:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000502b3e4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664352000:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189467.324888:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664352064 00000100:00100000:1.0:1776189467.324898:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10017 00000100:00100000:1.0:1776189467.324900:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000aa8befb6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664351616:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189467.324905:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189467.324929:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10018 00000100:00100000:0.0:1776189467.324932:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d4da5ec0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664352064:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189467.324934:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000aa8befb6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664351616:12345-0@lo:400:kworker/u8:1.0 Request processed in 33us (132us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189467.324939:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10017 00000100:00100000:1.0:1776189467.324951:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57417 00000100:00100000:1.0:1776189467.324954:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000028933edf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664351808:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189467.324957:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d4da5ec0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664352064:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (126us total) trans 0 rc 0/0 00010000:02000000:1.0:1776189467.324961:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189467.324962:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10018 00000100:00100000:1.0:1776189467.324979:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000028933edf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664351808:12345-0@lo:38:kworker/u8:1.0 Request processed in 24us (264us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189467.324985:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664351552 00000100:00100000:1.0:1776189467.324985:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57417 00000100:00100000:0.0:1776189467.325005:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1487 00000100:00100000:0.0:1776189467.325008:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004c446be7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664351552:12345-0@lo:400:kworker/u8:1.0 00000100:00080000:1.0:1776189467.325011:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189467.325013:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189467.325016:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005be3c9f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664351808:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776189467.325025:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189467.325027:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189467.325029:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002f5aeefd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664351680:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189467.325032:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004c446be7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664351552:12345-0@lo:400:kworker/u8:1.0 Request processed in 23us (160us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189467.325037:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1487 00000100:00100000:1.0:1776189467.325056:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d5cfaed8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664351552:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189467.325066:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dfad713d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664351616:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189467.325067:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d01df670 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664351744:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189467.325073:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a943cb24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664352064:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189467.325086:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189467.325105:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776189467.325109:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc4b762b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664351936:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189467.325116:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664351744 00000100:00100000:0.0:1776189467.325129:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189467.325136:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10019 00000100:00100000:1.0:1776189467.325140:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a14e98d2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664351744:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189467.325150:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664352000 00000100:00100000:1.0:1776189467.325170:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a14e98d2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664351744:12345-0@lo:400:kworker/u8:1.0 Request processed in 29us (83us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189467.325171:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664351936 00000100:00100000:1.0:1776189467.325195:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10019 00000100:00100000:0.0:1776189467.325204:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8602 00000100:00100000:0.0:1776189467.325207:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0b45fb4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664352000:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189467.325212:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8603 00000100:00100000:1.0:1776189467.325215:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000960dca04 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664351936:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189467.325273:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664351424 00000100:00100000:3.0:1776189467.325297:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664351488 00000100:00100000:3.0:1776189467.325309:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6858 00000100:00100000:3.0:1776189467.325313:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000862ba2d5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664351424:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776189467.325327:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189467.325419:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664351872 00000100:00100000:2.0:1776189467.325441:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9138 00000100:00100000:2.0:1776189467.325445:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000797ba66f pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664351872:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189467.325453:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189467.334986:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000797ba66f pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664351872:12345-0@lo:38:kworker/u8:1.0 Request processed in 9531us (10228us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189467.335003:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9138 00000100:00100000:2.0:1776189467.335054:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6859 00000100:00100000:2.0:1776189467.335061:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000adb1e111 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664351488:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189467.335088:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189467.336089:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000960dca04 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664351936:12345-0@lo:400:kworker/u8:1.0 Request processed in 10870us (10956us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189467.336101:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8603 00000100:00080000:1.0:1776189467.336160:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189467.336165:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189467.336169:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007f770c5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664351872:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189467.336300:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0b45fb4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664352000:12345-0@lo:400:kworker/u8:1.0 Request processed in 11090us (11392us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189467.336308:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8602 00000100:00100000:0.0:1776189467.336374:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000502b3e4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664352000:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189467.337125:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000862ba2d5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664351424:12345-0@lo:13:osp-pre-0-1.0 Request processed in 11809us (12549us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189467.337135:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6858 00000100:00100000:2.0:1776189467.337544:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000adb1e111 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664351488:12345-0@lo:13:osp-pre-1-1.0 Request processed in 2482us (12910us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189467.337552:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6859 00000100:00100000:1.0:1776189467.342793:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc4b762b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664351936:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189467.342835:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d01df670 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664351744:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189467.342859:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005c6d79ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664351488:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189467.342874:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008ceb4933 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664351424:0@lo:13:osp-pre-0-1.0 00010000:02020000:2.0:1776189467.356897:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189467.613394:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189467.644663:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189467.868344:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189468.124271:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189468.380303:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189468.636368:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189468.668571:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189468.892299:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189469.148295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189469.405075:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189469.660389:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189469.692395:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189469.916297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189470.173208:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189470.429207:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189470.687222:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189470.717801:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189470.940293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189471.099562:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189471.099603:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189471.099824:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189471.100539:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654956800 00000100:00100000:2.0:1776189471.100595:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654956864 00000100:00100000:2.0:1776189471.100615:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10020 00000100:00100000:2.0:1776189471.100621:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ab2fefce pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654956800:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189471.100686:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ab2fefce pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654956800:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 65us (1083us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189471.100693:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10020 00000100:00100000:2.0:1776189471.100709:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57418 00000100:00100000:2.0:1776189471.100713:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008ceb4933 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654956864:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776189471.100726:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189471.100837:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008ceb4933 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654956864:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 123us (1014us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189471.100844:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57418 00000100:00100000:1.0:1776189471.101419:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654956736 00000100:00100000:1.0:1776189471.101470:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1488 00000100:00100000:1.0:1776189471.101474:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fe4b4330 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654956736:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189471.101507:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fe4b4330 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654956736:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 32us (1954us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189471.101514:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1488 00010000:02020000:2.0:1776189471.196296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189471.420460:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004dd53b72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664352128:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189471.420527:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189471.420627:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664352128 00000100:00100000:2.0:1776189471.420674:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6860 00000100:00100000:2.0:1776189471.420679:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000036a45663 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664352128:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189471.420705:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189471.420880:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005c6d79ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664352192:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189471.420908:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189471.420934:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664352192 00000100:00100000:3.0:1776189471.420957:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6861 00000100:00100000:3.0:1776189471.420960:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b696663d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664352192:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189471.420976:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189471.421016:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b696663d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664352192:12345-0@lo:13:osp-pre-0-0.0 Request processed in 54us (107us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189471.421022:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6861 00000100:00100000:3.0:1776189471.421078:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005c6d79ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664352192:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189471.421332:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000036a45663 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664352128:12345-0@lo:13:osp-pre-1-0.0 Request processed in 651us (805us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189471.421339:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6860 00000100:00100000:0.0:1776189471.421420:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004dd53b72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664352128:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189471.452295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189471.708356:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189471.740573:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189471.964347:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189472.220298:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189472.444459:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002477597b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664352256:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189472.444532:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776189472.444598:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189472.444603:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189472.444606:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189472.444610:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9614 00000100:00080000:1.0:1776189472.444614:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:1.0:1776189472.444619:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189472.444637:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007f770c5d x1862459664352512/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189472.444659:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189472.444663:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189472.444664:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189472.444667:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9614 00000100:00080000:1.0:1776189472.444669:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00080000:1.0:1776189472.444672:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189472.444681:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000dfad713d x1862459664352640/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189472.444688:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189472.444690:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189472.444691:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189472.444694:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9614 00000100:00080000:1.0:1776189472.444696:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00080000:1.0:1776189472.444698:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189472.444718:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d5cfaed8 x1862459664352704/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189472.444833:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a943cb24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664352576:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189472.444875:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189472.444882:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc4b762b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664352448:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189472.444898:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776189472.444906:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a45b748c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664352320:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189472.444917:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189472.444939:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776189472.444947:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d01df670 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664352384:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189472.444954:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189472.445053:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189472.445071:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005c6d79ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664352896:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189472.445159:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189472.445196:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776189472.445202:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005be3c9f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664352832:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189472.445238:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189472.445269:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664352576 00000100:00100000:3.0:1776189472.445307:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664352448 00000100:00100000:3.0:1776189472.445433:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664352896 00000100:00100000:3.0:1776189472.445451:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10021 00000100:00100000:3.0:1776189472.445456:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a4221496 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664352576:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189472.445500:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a4221496 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664352576:12345-0@lo:400:kworker/u8:1.0 Request processed in 43us (626us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189472.445507:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10021 00000100:00100000:3.0:1776189472.445523:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10022 00000100:00100000:3.0:1776189472.445527:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003da19f87 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664352448:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189472.445558:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003da19f87 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664352448:12345-0@lo:400:kworker/u8:1.0 Request processed in 29us (642us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189472.445564:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10022 00000100:00100000:3.0:1776189472.445575:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10023 00000100:00100000:3.0:1776189472.445578:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c995aa10 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664352896:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189472.445600:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664352256 00000100:00100000:3.0:1776189472.445605:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c995aa10 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664352896:12345-0@lo:400:kworker/u8:1.0 Request processed in 25us (446us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189472.445611:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10023 00000100:00100000:3.0:1776189472.445638:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664352832 00000100:00100000:0.0:1776189472.445642:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6862 00000100:00100000:0.0:1776189472.445648:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c3d6c201 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664352256:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776189472.445664:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189472.445668:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8604 00000100:00100000:3.0:1776189472.445674:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007bf8226c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664352832:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189472.445700:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c3d6c201 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664352256:12345-0@lo:13:osp-pre-0-1.0 Request processed in 52us (1170us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189472.445707:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6862 00000100:00100000:3.0:1776189472.445726:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007bf8226c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664352832:12345-0@lo:400:kworker/u8:1.0 Request processed in 50us (498us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189472.445730:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002f5aeefd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664352768:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189472.445736:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8604 00000100:00100000:3.0:1776189472.445762:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007f770c5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664352512:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189472.445764:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189472.445783:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776189472.445789:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000502b3e4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664352960:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189472.445809:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189472.445843:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664352768 00000100:00100000:0.0:1776189472.445864:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664352960 00000100:00100000:0.0:1776189472.445877:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8605 00000100:00100000:0.0:1776189472.445881:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000114609f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664352768:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189472.445913:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000114609f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664352768:12345-0@lo:400:kworker/u8:1.0 Request processed in 31us (153us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189472.445919:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8605 00000100:00100000:0.0:1776189472.445929:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8606 00000100:00100000:0.0:1776189472.445932:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001ce461df pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664352960:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189472.445955:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001ce461df pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664352960:12345-0@lo:400:kworker/u8:1.0 Request processed in 21us (145us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189472.445960:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8606 00000100:00100000:0.0:1776189472.445995:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000502b3e4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664352960:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189472.446005:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002f5aeefd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664352768:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189472.446063:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189472.446084:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dfad713d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664352640:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189472.446108:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189472.446127:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d5cfaed8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664352704:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189472.446136:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664352512 00000100:00100000:3.0:1776189472.446155:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189472.446156:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664352640 00000100:00100000:0.0:1776189472.446204:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57419 00000100:00100000:0.0:1776189472.446209:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000727dd03d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664352512:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189472.446218:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005be3c9f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664352832:0@lo:400:kworker/u8:1.0 00010000:02000000:0.0:1776189472.446224:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189472.446241:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005c6d79ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664352896:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189472.446255:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc4b762b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664352448:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189472.446280:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000727dd03d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664352512:12345-0@lo:38:kworker/u8:1.0 Request processed in 69us (216us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189472.446288:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57419 00000100:00100000:3.0:1776189472.446295:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57420 00000100:00100000:3.0:1776189472.446300:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000054fe574a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664352640:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189472.446309:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189472.446315:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664352704 00000100:00100000:0.0:1776189472.446342:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9139 00000100:00100000:3.0:1776189472.446342:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000054fe574a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664352640:12345-0@lo:38:kworker/u8:1.0 Request processed in 40us (234us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189472.446350:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000308dffba pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664352704:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189472.446354:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57420 00010000:02000000:0.0:1776189472.446360:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189472.446395:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664352384 00000100:00100000:0.0:1776189472.446405:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000308dffba pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664352704:12345-0@lo:38:kworker/u8:1.0 Request processed in 55us (252us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189472.446412:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9139 00000100:00100000:3.0:1776189472.446424:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1489 00000100:00100000:3.0:1776189472.446429:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000459e616f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664352384:12345-0@lo:400:kworker/u8:1.0 00000100:00080000:0.0:1776189472.446457:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189472.446461:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189472.446467:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007f770c5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664352512:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189472.446470:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000459e616f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664352384:12345-0@lo:400:kworker/u8:1.0 Request processed in 38us (1416us total) trans 0 rc 0/0 00000100:00080000:0.0:1776189472.446480:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:3.0:1776189472.446480:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1489 00000100:00080000:0.0:1776189472.446486:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189472.446491:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dfad713d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664352640:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776189472.446501:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189472.446504:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189472.446507:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d5cfaed8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664352704:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189472.446531:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d01df670 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664352384:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189472.446711:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a943cb24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664352576:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189472.446728:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002477597b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664352256:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189472.446864:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664352320 00000100:00100000:0.0:1776189472.446895:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6863 00000100:00100000:0.0:1776189472.446899:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006302d616 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664352320:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189472.446913:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189472.446944:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006302d616 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664352320:12345-0@lo:13:osp-pre-1-1.0 Request processed in 44us (1994us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189472.446951:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6863 00000100:00100000:2.0:1776189472.447019:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a45b748c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664352320:0@lo:13:osp-pre-1-1.0 00010000:02020000:2.0:1776189472.476949:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189472.733049:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189472.764498:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189472.990868:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189473.244394:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189473.500266:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189473.756297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189473.789408:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189474.012483:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189474.268280:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189474.524475:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189474.780779:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189474.812483:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189475.036317:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189475.292400:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189475.548368:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189475.804743:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189475.836468:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189476.060337:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189476.219843:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189476.219890:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189476.219922:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189476.219971:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654957056 00000100:00100000:2.0:1776189476.219983:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654956992 00000100:00100000:1.0:1776189476.220021:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654956928 00000100:00100000:3.0:1776189476.220022:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57421 00000100:00100000:3.0:1776189476.220033:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008809b423 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654957056:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:3.0:1776189476.220052:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189476.220062:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1490 00000100:00100000:2.0:1776189476.220062:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10024 00000100:00100000:1.0:1776189476.220069:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000de62cc6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654956928:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189476.220069:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009ac11ef3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654956992:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189476.220119:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008809b423 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654957056:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 84us (228us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189476.220128:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57421 00000100:00100000:1.0:1776189476.220157:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000de62cc6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654956928:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 76us (309us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189476.220167:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1490 00000100:00100000:2.0:1776189476.221120:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009ac11ef3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654956992:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 1045us (1192us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189476.221133:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10024 00010000:02020000:2.0:1776189476.317318:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189476.540573:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a943cb24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664353088:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189476.540585:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776189476.540612:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b9ada007 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664353024:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189476.540655:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189476.540937:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664353088 00000100:00100000:1.0:1776189476.541011:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6864 00000100:00100000:1.0:1776189476.541018:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000270ed845 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664353088:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189476.541057:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189476.544620:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189476.544727:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664353024 00000100:00100000:0.0:1776189476.544803:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6865 00000100:00100000:0.0:1776189476.544810:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a45b748c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664353024:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189476.544862:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189476.544949:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a45b748c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664353024:12345-0@lo:13:osp-pre-1-0.0 Request processed in 138us (339us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189476.544957:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6865 00000100:00100000:2.0:1776189476.545082:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b9ada007 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664353024:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189476.546956:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000270ed845 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664353088:12345-0@lo:13:osp-pre-0-0.0 Request processed in 5927us (6293us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189476.546976:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6864 00000100:00100000:3.0:1776189476.548884:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a943cb24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664353088:0@lo:13:osp-pre-0-0.0 00010000:02020000:2.0:1776189476.572922:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189476.830682:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189476.866218:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189477.084302:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189477.340747:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189477.564439:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed29900a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664353152:0@lo:13:osp-pre-1-1.0 00000100:00080000:0.0:1776189477.564529:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189477.564535:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189477.564539:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189477.564550:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9619 00000100:00100000:1.0:1776189477.564559:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776189477.564560:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1776189477.564572:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189477.564596:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a38efcb6 x1862459664353344/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189477.564638:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189477.564641:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189477.564642:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1776189477.564643:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664353152 00000100:00080000:0.0:1776189477.564660:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9619 00000100:00080000:0.0:1776189477.564664:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1776189477.564668:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189477.564685:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004dd53b72 x1862459664353472/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189477.564694:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189477.564698:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189477.564701:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1776189477.564703:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6866 00000100:00080000:0.0:1776189477.564707:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9619 00000100:00080000:0.0:1776189477.564713:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1776189477.564717:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d01df670 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664353152:12345-0@lo:13:osp-pre-1-1.0 00000100:00080000:0.0:1776189477.564721:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189477.564741:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001b0d11fc x1862459664353536/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00002000:00000024:1.0:1776189477.564755:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189477.564808:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d01df670 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664353152:12345-0@lo:13:osp-pre-1-1.0 Request processed in 95us (256us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189477.564812:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f2a9137 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664353280:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189477.564822:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6866 00000100:00100000:1.0:1776189477.564857:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000880386e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664353600:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189477.564894:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189477.564904:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189477.564933:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c921b786 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664353728:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189477.564965:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed29900a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664353152:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189477.564984:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189477.565012:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1776189477.565019:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001d029ec5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664353216:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189477.565048:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189477.565065:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a969b6e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664353664:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189477.565089:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189477.565124:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a38efcb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664353344:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189477.565147:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189477.565159:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004dd53b72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664353472:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189477.565210:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189477.565227:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001b0d11fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664353536:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189477.565255:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189477.565256:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1776189477.565270:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7c1f715 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664353408:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189477.565301:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664353600 00000100:00100000:1.0:1776189477.565312:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189477.565339:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c731e56f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664353792:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189477.565349:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664353664 00000100:00100000:1.0:1776189477.565369:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189477.565378:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8607 00000100:00100000:0.0:1776189477.565385:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d5cfaed8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664353600:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189477.565414:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664353792 00000100:00100000:0.0:1776189477.565461:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d5cfaed8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664353600:12345-0@lo:400:kworker/u8:0.0 Request processed in 66us (551us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189477.565461:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8608 00000100:00100000:1.0:1776189477.565471:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000375d4d18 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664353664:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189477.565480:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8607 00000100:00100000:0.0:1776189477.565508:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8609 00000100:00100000:0.0:1776189477.565515:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007f770c5d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664353792:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189477.565538:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000375d4d18 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664353664:12345-0@lo:400:kworker/u8:0.0 Request processed in 67us (447us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189477.565549:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8608 00000100:00100000:0.0:1776189477.565569:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007f770c5d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664353792:12345-0@lo:400:kworker/u8:0.0 Request processed in 52us (199us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189477.565578:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8609 00000100:00100000:1.0:1776189477.565589:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664353216 00000100:00100000:1.0:1776189477.565633:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1491 00000100:00100000:1.0:1776189477.565638:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000082b19b4e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664353216:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189477.565645:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a969b6e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664353664:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189477.565688:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000082b19b4e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664353216:12345-0@lo:400:kworker/u8:0.0 Request processed in 47us (637us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189477.565697:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1491 00000100:00100000:0.0:1776189477.565701:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001d029ec5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664353216:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189477.565802:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664353280 00000100:00100000:2.0:1776189477.565833:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664353728 00000100:00100000:2.0:1776189477.565842:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664353344 00000100:00100000:2.0:1776189477.565849:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664353472 00000100:00100000:2.0:1776189477.565855:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664353408 00000100:00100000:2.0:1776189477.565865:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10025 00000100:00100000:2.0:1776189477.565868:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b3b3d75f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664353280:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189477.565991:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10026 00000100:00100000:3.0:1776189477.565995:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006f1e79e6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664353728:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189477.566152:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd069f70 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664353856:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189477.566210:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189477.566236:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c731e56f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664353792:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189477.566245:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000880386e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664353600:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189477.566283:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664353536 00000100:00100000:0.0:1776189477.566301:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9140 00000100:00100000:0.0:1776189477.566304:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009b5045f9 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664353536:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189477.566318:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189477.566348:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009b5045f9 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664353536:12345-0@lo:38:kworker/u8:0.0 Request processed in 43us (1097us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189477.566354:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9140 00000100:00100000:0.0:1776189477.566372:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10027 00000100:00100000:0.0:1776189477.566376:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dfad713d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664353408:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189477.566411:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dfad713d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664353408:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (1100us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189477.566417:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10027 00000100:00100000:0.0:1776189477.566432:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57422 00000100:00100000:0.0:1776189477.566435:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002696d473 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664353344:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189477.566442:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189477.566464:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002696d473 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664353344:12345-0@lo:38:kworker/u8:0.0 Request processed in 28us (1317us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189477.566469:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57422 00000100:00100000:0.0:1776189477.566480:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57423 00000100:00100000:0.0:1776189477.566483:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ddffdaa5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664353472:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189477.566489:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189477.566508:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ddffdaa5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664353472:12345-0@lo:38:kworker/u8:0.0 Request processed in 24us (1299us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189477.566513:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57423 00000100:00080000:0.0:1776189477.566549:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189477.566552:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189477.566555:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a38efcb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664353344:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776189477.566563:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189477.566566:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189477.566568:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004dd53b72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664353472:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776189477.566575:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189477.566577:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189477.566579:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001b0d11fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664353536:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189477.566617:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7c1f715 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664353408:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189477.566666:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664353856 00000100:00100000:0.0:1776189477.569238:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6867 00000100:00100000:0.0:1776189477.569249:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a969b6e4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664353856:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776189477.569281:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189477.569375:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a969b6e4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664353856:12345-0@lo:13:osp-pre-0-1.0 Request processed in 113us (3151us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189477.569383:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6867 00000100:00100000:0.0:1776189477.569442:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd069f70 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664353856:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189477.570311:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b3b3d75f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664353280:12345-0@lo:400:kworker/u8:0.0 Request processed in 4438us (5416us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189477.570321:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10025 00000100:00100000:3.0:1776189477.570698:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006f1e79e6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664353728:12345-0@lo:400:kworker/u8:0.0 Request processed in 4701us (5728us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189477.570707:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10026 00000100:00100000:0.0:1776189477.573246:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f2a9137 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664353280:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189477.573278:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c921b786 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664353728:0@lo:400:kworker/u8:0.0 00010000:02020000:2.0:1776189477.596537:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189477.852314:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189477.884471:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189478.108462:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189478.364728:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189478.620988:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189478.876335:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189478.910378:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189479.132300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189479.388545:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189479.644360:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189479.900347:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189479.932661:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189480.156809:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189480.412356:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189480.668414:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189480.926442:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189480.956524:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189481.181519:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189481.340535:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189481.340592:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189481.340678:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654957184 00000100:00100000:3.0:1776189481.340704:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189481.340736:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654957248 00000100:00100000:1.0:1776189481.340755:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10028 00000100:00100000:1.0:1776189481.340763:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000136e6642 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654957184:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1776189481.340772:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57424 00000100:00100000:3.0:1776189481.340778:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000997c7cab pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654957248:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:3.0:1776189481.340794:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189481.340829:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000136e6642 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654957184:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 66us (303us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189481.340838:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10028 00000100:00100000:2.0:1776189481.341982:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654957120 00000100:00100000:2.0:1776189481.342029:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1492 00000100:00100000:2.0:1776189481.342043:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000055012753 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654957120:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1776189481.342536:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000997c7cab pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654957248:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 1755us (1832us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189481.342546:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57424 00000100:00100000:2.0:1776189481.347171:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000055012753 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654957120:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 5126us (6570us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189481.347211:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1492 00010000:02020000:2.0:1776189481.436314:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189481.660788:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc4b762b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664353920:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189481.660820:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004088ae05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664353984:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189481.660859:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189481.661039:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189481.661124:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664353920 00000100:00100000:3.0:1776189481.661124:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664353984 00000100:00100000:3.0:1776189481.661189:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6868 00000100:00100000:3.0:1776189481.661195:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c921b786 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664353984:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776189481.661221:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189481.663132:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6869 00000100:00100000:2.0:1776189481.663147:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003cd25a35 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664353920:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189481.663192:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189481.663259:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003cd25a35 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664353920:12345-0@lo:13:osp-pre-0-0.0 Request processed in 114us (2401us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189481.663266:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6869 00000100:00100000:2.0:1776189481.663326:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc4b762b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664353920:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189481.671335:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c921b786 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664353984:12345-0@lo:13:osp-pre-1-0.0 Request processed in 10129us (10299us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189481.671352:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6868 00000100:00100000:0.0:1776189481.674055:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004088ae05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664353984:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189481.693526:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189481.948334:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189481.980582:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189482.204745:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189482.460279:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:0.0:1776189482.684720:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189482.684727:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189482.684730:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189482.684738:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9624 00000100:00080000:0.0:1776189482.684741:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1776189482.684746:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189482.684765:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c731e56f x1862459664354176/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189482.684800:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189482.684805:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189482.684806:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189482.684810:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9624 00000100:00080000:0.0:1776189482.684812:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1776189482.684815:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189482.684827:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c731e56f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664354176:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189482.684828:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000f2a9137 x1862459664354368/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189482.684841:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189482.684843:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189482.684845:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189482.684848:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9624 00000100:00080000:0.0:1776189482.684849:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1776189482.684852:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189482.684864:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c7c1f715 x1862459664354432/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189482.684897:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189482.684914:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004dd53b72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664354304:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189482.684941:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f2a9137 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664354368:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189482.684948:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189482.684970:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a8591738 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664354688:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189482.684976:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189482.684996:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7c1f715 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664354432:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189482.685015:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189482.685038:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189482.685039:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1776189482.685046:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e48fdf91 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664354240:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189482.685071:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189482.685082:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664354176 00000100:00100000:3.0:1776189482.685113:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664354304 00000100:00100000:3.0:1776189482.685137:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664354368 00000100:00100000:3.0:1776189482.685152:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10029 00000100:00100000:3.0:1776189482.685157:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000079c74b68 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664354304:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189482.685294:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000880386e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664354112:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189482.685344:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd150e5b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664354624:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189482.685413:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001b0d11fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664354048:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189482.685870:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189482.685883:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001d029ec5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664354560:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189482.685904:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189482.685952:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664354688 00000100:00100000:0.0:1776189482.686561:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189482.686593:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664354560 00000100:00100000:0.0:1776189482.686624:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8610 00000100:00100000:0.0:1776189482.686628:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000087b82cd7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664354688:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189482.686672:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000087b82cd7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664354688:12345-0@lo:400:kworker/u8:1.0 Request processed in 43us (1668us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189482.686679:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8610 00000100:00100000:2.0:1776189482.686837:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8611 00000100:00100000:2.0:1776189482.686843:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c68cc49e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664354560:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189482.687363:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664354432 00000100:00100000:0.0:1776189482.687387:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9141 00000100:00100000:0.0:1776189482.687391:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b9ada007 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664354432:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189482.687406:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189482.687447:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b9ada007 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664354432:12345-0@lo:38:kworker/u8:1.0 Request processed in 54us (2410us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189482.687454:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9141 00000100:00100000:2.0:1776189482.687794:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c68cc49e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664354560:12345-0@lo:400:kworker/u8:1.0 Request processed in 41us (979us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189482.687802:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8611 00000100:00100000:2.0:1776189482.687833:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664354112 00000100:00100000:2.0:1776189482.687853:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664354624 00000100:00100000:2.0:1776189482.687865:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10030 00000100:00100000:2.0:1776189482.687870:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000032d9de2c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664354112:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189482.687908:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189482.687929:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000032d9de2c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664354112:12345-0@lo:400:kworker/u8:1.0 Request processed in 57us (2059us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189482.687937:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10030 00000100:00100000:1.0:1776189482.687939:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a38efcb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664354496:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189482.687954:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10031 00000100:00100000:2.0:1776189482.687959:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000058a2b052 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664354624:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189482.687970:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189482.687999:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000058a2b052 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664354624:12345-0@lo:400:kworker/u8:1.0 Request processed in 38us (1438us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189482.688411:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10031 00000100:00100000:0.0:1776189482.688449:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57425 00000100:00100000:0.0:1776189482.688455:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004088ae05 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664354176:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189482.688468:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189482.688500:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004088ae05 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664354176:12345-0@lo:38:kworker/u8:1.0 Request processed in 45us (3605us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189482.688506:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57425 00000100:00100000:0.0:1776189482.688521:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57426 00000100:00100000:0.0:1776189482.688524:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cd069f70 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664354368:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189482.688531:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189482.688551:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cd069f70 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664354368:12345-0@lo:38:kworker/u8:1.0 Request processed in 26us (3578us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189482.688557:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57426 00000100:00100000:0.0:1776189482.688575:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664354496 00000100:00100000:0.0:1776189482.688603:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8612 00000100:00100000:0.0:1776189482.688606:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005c6d79ae pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664354496:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189482.688639:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005c6d79ae pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664354496:12345-0@lo:400:kworker/u8:1.0 Request processed in 31us (670us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189482.688644:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8612 00000100:00100000:2.0:1776189482.688694:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000880386e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664354112:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189482.688707:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001d029ec5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664354560:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189482.688766:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664354240 00000100:00100000:2.0:1776189482.688789:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6870 00000100:00100000:2.0:1776189482.688792:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005d995c27 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664354240:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189482.688806:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189482.688841:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005d995c27 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664354240:12345-0@lo:13:osp-pre-0-1.0 Request processed in 48us (3771us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189482.688847:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6870 00000100:00100000:1.0:1776189482.688870:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007c850b41 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664354752:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189482.688904:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1776189482.688911:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189482.688916:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189482.688921:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f2a9137 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664354368:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776189482.688937:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:1.0:1776189482.688938:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a38efcb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664354496:0@lo:400:kworker/u8:1.0 00000100:00080000:2.0:1776189482.688941:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189482.688947:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7c1f715 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664354432:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776189482.688960:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189482.688963:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189482.688966:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c731e56f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664354176:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189482.689483:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd150e5b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664354624:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189482.689505:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a8591738 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664354688:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189482.689533:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e48fdf91 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664354240:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189482.689683:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664354048 00000100:00100000:1.0:1776189482.690224:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664354752 00000100:00100000:1.0:1776189482.690262:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8613 00000100:00100000:1.0:1776189482.690267:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005be3c9f3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664354752:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189482.690327:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005be3c9f3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664354752:12345-0@lo:400:kworker/u8:1.0 Request processed in 59us (1423us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189482.690333:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8613 00000100:00100000:1.0:1776189482.690362:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002f5aeefd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664354816:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189482.690420:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000079c74b68 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664354304:12345-0@lo:400:kworker/u8:1.0 Request processed in 5260us (5473us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189482.690429:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10029 00000100:00100000:1.0:1776189482.690540:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189482.690656:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007c850b41 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664354752:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189482.690777:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664354816 00000100:00100000:3.0:1776189482.690848:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6871 00000100:00100000:3.0:1776189482.690852:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000502b3e4a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664354816:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189482.690866:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189482.690909:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000502b3e4a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664354816:12345-0@lo:13:osp-pre-1-1.0 Request processed in 56us (512us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189482.690916:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6871 00000100:00100000:2.0:1776189482.691040:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1493 00000100:00100000:0.0:1776189482.691046:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004dd53b72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664354304:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189482.691050:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bffe4051 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664354048:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189482.691116:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002f5aeefd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664354816:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189482.691197:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bffe4051 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664354048:12345-0@lo:400:kworker/u8:1.0 Request processed in 149us (3290us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189482.691204:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1493 00000100:00100000:0.0:1776189482.691472:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001b0d11fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664354048:0@lo:400:kworker/u8:1.0 00010000:02020000:2.0:1776189482.717033:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189482.972483:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189483.006063:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189483.228458:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189483.484335:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189483.740301:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189483.996282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189484.029573:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189484.252473:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189484.509237:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189484.764412:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189485.020307:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189485.052538:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189485.276346:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189485.532442:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189485.788293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189486.044373:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189486.076457:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189486.300911:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189486.458276:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189486.458332:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189486.458385:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189486.458479:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654957440 00000100:00100000:0.0:1776189486.458528:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654957376 00000100:00100000:1.0:1776189486.458560:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10032 00000100:00100000:1.0:1776189486.458568:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e48fdf91 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654957376:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776189486.458665:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e48fdf91 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654957376:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 97us (281us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189486.458735:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654957312 00000100:00100000:2.0:1776189486.458767:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1494 00000100:00100000:2.0:1776189486.458772:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002aae1640 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654957312:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776189486.458799:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002aae1640 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654957312:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 27us (468us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189486.458810:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1494 00000100:00100000:0.0:1776189486.461134:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57427 00000100:00100000:0.0:1776189486.461146:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002477597b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654957440:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:0.0:1776189486.461167:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189486.461273:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002477597b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654957440:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 128us (3005us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189486.461281:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57427 00000100:00100000:1.0:1776189486.465759:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10032 00010000:02020000:0.0:1776189486.556946:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189486.780553:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776189486.780587:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae1dab54 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664354880:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189486.780656:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189486.781110:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002f5aeefd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664354944:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189486.781138:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189486.781273:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664354880 00000100:00100000:2.0:1776189486.781318:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664354944 00000100:00100000:2.0:1776189486.781340:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6872 00000100:00100000:2.0:1776189486.781346:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f0b13a19 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664354880:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189486.781378:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189486.781425:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f0b13a19 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664354880:12345-0@lo:13:osp-pre-1-0.0 Request processed in 78us (769us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189486.781432:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6872 00000100:00100000:2.0:1776189486.781451:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6873 00000100:00100000:2.0:1776189486.781455:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f6664e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664354944:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189486.781472:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189486.781501:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f6664e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664354944:12345-0@lo:13:osp-pre-0-0.0 Request processed in 45us (363us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189486.781508:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6873 00000100:00100000:2.0:1776189486.781585:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002f5aeefd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664354944:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189486.781602:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae1dab54 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664354880:0@lo:13:osp-pre-1-0.0 00010000:02020000:0.0:1776189486.812494:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189487.070580:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189487.101200:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189487.324292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189487.580957:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189487.804485:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776189487.804509:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc4b762b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664355008:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189487.804574:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189487.804731:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664355008 00000100:00100000:1.0:1776189487.804783:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6874 00000100:00100000:1.0:1776189487.804788:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000038a7e045 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664355008:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189487.804810:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189487.804854:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000038a7e045 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664355008:12345-0@lo:13:osp-pre-1-1.0 Request processed in 65us (281us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189487.804861:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6874 00000100:00100000:1.0:1776189487.804918:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc4b762b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664355008:0@lo:13:osp-pre-1-1.0 00000100:00080000:2.0:1776189487.805147:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189487.805152:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189487.805155:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189487.805160:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9629 00000100:00080000:2.0:1776189487.805162:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776189487.805165:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189487.805201:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000f2a9137 x1862459664355200/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189487.805226:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189487.805229:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189487.805231:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189487.805234:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9629 00000100:00080000:2.0:1776189487.805236:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776189487.805239:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189487.805248:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001d029ec5 x1862459664355328/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189487.805255:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189487.805258:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189487.805259:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189487.805261:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9629 00000100:00080000:2.0:1776189487.805263:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776189487.805266:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189487.805278:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001b0d11fc x1862459664355392/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189487.805322:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7c1f715 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664355136:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189487.805351:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189487.805366:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d3414bd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664355584:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189487.805386:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189487.805403:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1776189487.805408:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a38efcb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664355264:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189487.805430:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189487.805441:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001a12eabb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664355648:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189487.805794:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c731e56f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664355456:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189487.805850:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed29900a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664355072:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189487.805982:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664355136 00000100:00100000:3.0:1776189487.806015:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664355584 00000100:00100000:3.0:1776189487.806031:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664355264 00000100:00100000:3.0:1776189487.806048:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10033 00000100:00100000:3.0:1776189487.806052:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000481714e2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664355136:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189487.806148:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189487.806190:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000032f89d8f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664355712:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189487.806214:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189487.806242:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10034 00000100:00100000:0.0:1776189487.806246:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fa770ed8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664355584:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189487.806276:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fa770ed8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664355584:12345-0@lo:400:kworker/u8:1.0 Request processed in 29us (890us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189487.806283:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10034 00000100:00100000:0.0:1776189487.806299:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10035 00000100:00100000:0.0:1776189487.806302:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b3632b78 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664355264:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189487.806328:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b3632b78 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664355264:12345-0@lo:400:kworker/u8:1.0 Request processed in 25us (898us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189487.806334:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10035 00000100:00100000:0.0:1776189487.806355:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664355456 00000100:00100000:0.0:1776189487.806379:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8614 00000100:00100000:0.0:1776189487.806382:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004dd53b72 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664355456:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189487.806413:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004dd53b72 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664355456:12345-0@lo:400:kworker/u8:1.0 Request processed in 30us (266us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189487.806419:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8614 00000100:00100000:0.0:1776189487.806448:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c731e56f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664355456:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189487.806542:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189487.806561:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000880386e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664355520:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189487.806566:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189487.806582:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189487.806588:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d3414bd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664355584:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189487.806614:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664355072 00000100:00100000:2.0:1776189487.806622:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a38efcb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664355264:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189487.806654:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f2a9137 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664355200:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189487.806677:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189487.806691:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001d029ec5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664355328:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189487.806710:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189487.806723:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001b0d11fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664355392:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189487.806744:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189487.806773:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664355200 00000100:00100000:2.0:1776189487.806788:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664355328 00000100:00100000:2.0:1776189487.806797:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57428 00000100:00100000:2.0:1776189487.806801:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d50ac393 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664355200:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189487.806814:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189487.806848:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d50ac393 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664355200:12345-0@lo:38:kworker/u8:1.0 Request processed in 46us (170us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189487.806851:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1495 00000100:00100000:2.0:1776189487.806854:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57428 00000100:00100000:1.0:1776189487.806855:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006f66ddc9 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664355072:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189487.806865:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57429 00000100:00100000:2.0:1776189487.806867:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004afd2049 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664355328:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189487.806874:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189487.806888:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006f66ddc9 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664355072:12345-0@lo:400:kworker/u8:1.0 Request processed in 32us (347us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189487.806890:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004afd2049 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664355328:12345-0@lo:38:kworker/u8:1.0 Request processed in 22us (181us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189487.806995:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57429 00000100:00080000:2.0:1776189487.807022:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189487.807026:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189487.807030:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f2a9137 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664355200:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776189487.807037:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189487.807039:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189487.807042:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001d029ec5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664355328:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189487.807092:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664355392 00000100:00100000:2.0:1776189487.807110:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9142 00000100:00100000:2.0:1776189487.807113:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a38efcb6 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664355392:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189487.807120:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189487.807150:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a38efcb6 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664355392:12345-0@lo:38:kworker/u8:1.0 Request processed in 35us (405us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189487.807156:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9142 00000100:00100000:1.0:1776189487.807169:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1495 00000100:00100000:2.0:1776189487.807188:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664355712 00000100:00100000:2.0:1776189487.807209:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6875 00000100:00100000:2.0:1776189487.807213:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007c850b41 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664355712:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189487.807216:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664355648 00002000:00000024:2.0:1776189487.807226:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189487.807232:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664355520 00000100:00100000:1.0:1776189487.807247:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8615 00000100:00100000:1.0:1776189487.807251:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ad3c7e22 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+6:3361:x1862459664355648:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189487.807253:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007c850b41 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3363:x1862459664355712:12345-0@lo:13:osp-pre-0-1.0 Request processed in 40us (1040us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189487.807259:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6875 00000100:00080000:2.0:1776189487.807279:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:1.0:1776189487.807281:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ad3c7e22 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664355648:12345-0@lo:400:kworker/u8:1.0 Request processed in 30us (715us total) trans 0 rc 0/0 00000100:00080000:2.0:1776189487.807283:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189487.807285:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001b0d11fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664355392:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189487.807288:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8615 00000100:00100000:1.0:1776189487.807297:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8616 00000100:00100000:1.0:1776189487.807301:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cc4b762b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664355520:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189487.807361:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001a12eabb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664355648:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189487.807423:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cc4b762b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664355520:12345-0@lo:400:kworker/u8:1.0 Request processed in 120us (840us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189487.807430:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8616 00000100:00100000:1.0:1776189487.807457:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed29900a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664355072:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189487.807466:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000880386e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664355520:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189487.807589:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000481714e2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664355136:12345-0@lo:400:kworker/u8:1.0 Request processed in 1534us (2237us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189487.807596:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000032f89d8f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664355712:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189487.807596:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10033 00000100:00100000:2.0:1776189487.807710:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7c1f715 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664355136:0@lo:400:kworker/u8:1.0 00010000:02020000:2.0:1776189487.840791:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189488.093359:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189488.127918:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189488.348449:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189488.604501:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189488.860339:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189489.119296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189489.152535:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189489.372339:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189489.629314:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189489.884358:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189490.140282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189490.172854:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189490.396370:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189490.654614:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189490.908513:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189491.164905:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189491.197470:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189491.420329:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189491.578811:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189491.578868:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189491.578981:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654957568 00000100:00100000:2.0:1776189491.579038:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654957632 00000100:00100000:2.0:1776189491.579060:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10036 00000100:00100000:2.0:1776189491.579068:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000032f89d8f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654957568:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776189491.579146:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000032f89d8f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654957568:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 78us (343us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189491.579154:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10036 00000100:00100000:2.0:1776189491.579204:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57430 00000100:00100000:2.0:1776189491.579209:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ae1dab54 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654957632:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:2.0:1776189491.580085:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189491.580134:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ae1dab54 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654957632:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 924us (1266us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189491.580143:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57430 00000100:00100000:3.0:1776189491.588777:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189491.589088:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654957504 00000100:00100000:1.0:1776189491.589150:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1496 00000100:00100000:1.0:1776189491.589157:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000002b5a0b3 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654957504:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776189491.589271:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000002b5a0b3 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654957504:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 113us (502us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189491.589279:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1496 00010000:02020000:2.0:1776189491.678401:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189491.902789:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a943cb24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664355840:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189491.902870:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189491.903028:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664355840 00000100:00100000:1.0:1776189491.903085:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6876 00000100:00100000:1.0:1776189491.903092:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000748b663e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664355840:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189491.903119:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189491.903162:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000748b663e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664355840:12345-0@lo:13:osp-pre-0-0.0 Request processed in 70us (293us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189491.903170:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6876 00000100:00100000:1.0:1776189491.903242:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a943cb24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664355840:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189491.903323:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776189491.903335:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000733a9e44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664355776:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189491.903364:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189491.903568:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664355776 00000100:00100000:1.0:1776189491.903600:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6877 00000100:00100000:1.0:1776189491.903604:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a8591738 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664355776:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189491.903622:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189491.903655:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a8591738 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664355776:12345-0@lo:13:osp-pre-1-0.0 Request processed in 50us (291us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189491.903661:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6877 00000100:00100000:1.0:1776189491.903701:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000733a9e44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664355776:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189491.932283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189492.188308:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189492.220427:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189492.445932:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189492.700293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189492.926203:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776189492.926236:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001706ae06 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664355904:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189492.926367:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f09d90e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664355968:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189492.926428:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189492.926480:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664355968 00000100:00100000:3.0:1776189492.926524:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6878 00000100:00100000:3.0:1776189492.926529:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007a543fbe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664355968:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189492.926553:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189492.926606:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007a543fbe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664355968:12345-0@lo:13:osp-pre-1-1.0 Request processed in 77us (179us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189492.926612:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6878 00000100:00100000:3.0:1776189492.926654:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f09d90e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664355968:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189492.926844:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189492.926913:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664355904 00000100:00100000:0.0:1776189492.926946:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6879 00000100:00100000:0.0:1776189492.926950:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a943cb24 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664355904:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776189492.926963:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189492.927012:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a943cb24 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664355904:12345-0@lo:13:osp-pre-0-1.0 Request processed in 61us (168us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189492.927018:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6879 00000100:00100000:1.0:1776189492.927200:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001706ae06 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664355904:0@lo:13:osp-pre-0-1.0 00000100:00080000:3.0:1776189492.927417:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189492.927422:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189492.927425:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189492.927430:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9634 00000100:00080000:3.0:1776189492.927433:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00080000:3.0:1776189492.927437:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189492.927459:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000011679d4d x1862459664356160/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189492.927487:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189492.927490:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189492.927492:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189492.927495:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9634 00000100:00080000:3.0:1776189492.927497:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00080000:3.0:1776189492.927500:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189492.927511:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000074a40efb x1862459664356288/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189492.927518:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189492.927521:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189492.927523:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189492.927526:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9634 00000100:00080000:3.0:1776189492.927528:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 6s 00000100:00080000:3.0:1776189492.927530:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189492.927540:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d9eb44b4 x1862459664356352/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189492.927686:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006235ac3c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664356096:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189492.927729:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189492.927758:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e42142fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664356544:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189492.927777:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189492.927803:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776189492.927808:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002ed2a919 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664356224:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189492.927829:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189492.927891:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664356096 00000100:00100000:3.0:1776189492.927931:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664356544 00000100:00100000:3.0:1776189492.927945:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664356224 00000100:00100000:3.0:1776189492.927959:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10037 00000100:00100000:3.0:1776189492.927963:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002f5aeefd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664356096:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189492.928013:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002f5aeefd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664356096:12345-0@lo:400:kworker/u8:1.0 Request processed in 49us (284us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189492.928020:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10037 00000100:00100000:3.0:1776189492.928035:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10038 00000100:00100000:3.0:1776189492.928039:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ed29900a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664356544:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189492.928063:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ed29900a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664356544:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (286us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189492.928068:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10038 00000100:00100000:3.0:1776189492.928079:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10039 00000100:00100000:3.0:1776189492.928082:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000042c1acfd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664356224:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189492.928105:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000042c1acfd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664356224:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (276us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189492.928110:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10039 00000100:00100000:2.0:1776189492.928262:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000011679d4d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664356160:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189492.928291:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189492.928311:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000074a40efb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664356288:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189492.928331:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189492.928345:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d9eb44b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664356352:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189492.928369:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189492.928398:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664356160 00000100:00100000:2.0:1776189492.928412:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664356288 00000100:00100000:2.0:1776189492.928424:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57431 00000100:00100000:2.0:1776189492.928427:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c7c1f715 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664356160:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189492.928440:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189492.928472:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c7c1f715 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664356160:12345-0@lo:38:kworker/u8:1.0 Request processed in 44us (181us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189492.928478:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57431 00000100:00100000:2.0:1776189492.928491:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57432 00000100:00100000:2.0:1776189492.928494:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001d029ec5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664356288:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189492.928501:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189492.928523:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001d029ec5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664356288:12345-0@lo:38:kworker/u8:1.0 Request processed in 28us (193us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189492.928529:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57432 00000100:00100000:2.0:1776189492.928555:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664356352 00000100:00100000:3.0:1776189492.928757:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f09d90e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664356032:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189492.928796:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189492.928812:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9c407a2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664356480:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189492.928833:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189492.928846:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776189492.928851:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e0bab4ab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664356416:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189492.928867:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189492.928946:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664356480 00000100:00100000:1.0:1776189492.928989:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664356416 00000100:00100000:1.0:1776189492.929006:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8617 00000100:00100000:1.0:1776189492.929009:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000089232900 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664356480:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189492.929047:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000089232900 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664356480:12345-0@lo:400:kworker/u8:1.0 Request processed in 36us (213us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189492.929052:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8617 00000100:00100000:1.0:1776189492.929064:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8618 00000100:00100000:1.0:1776189492.929067:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000031b5a624 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664356416:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189492.929090:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000031b5a624 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664356416:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (222us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189492.929095:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8618 00000100:00100000:1.0:1776189492.929134:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e0bab4ab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664356416:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189492.929144:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9c407a2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664356480:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189492.929197:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002ed2a919 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664356224:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189492.929208:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006235ac3c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664356096:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189492.929217:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e42142fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664356544:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189492.929254:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664356032 00000100:00100000:1.0:1776189492.929276:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1497 00000100:00100000:1.0:1776189492.929280:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000064f5974 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664356032:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189492.929310:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000064f5974 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664356032:12345-0@lo:400:kworker/u8:1.0 Request processed in 29us (513us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189492.929315:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1497 00000100:00100000:1.0:1776189492.929343:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f09d90e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664356032:0@lo:400:kworker/u8:1.0 00000100:00080000:1.0:1776189492.929380:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189492.929383:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189492.929387:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000011679d4d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664356160:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776189492.929394:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189492.929396:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189492.929399:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000074a40efb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664356288:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189492.930062:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9143 00000100:00100000:2.0:1776189492.930071:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000f2a9137 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664356352:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189492.930084:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189492.930137:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000f2a9137 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664356352:12345-0@lo:38:kworker/u8:1.0 Request processed in 67us (1767us total) trans 0 rc -19/-19 00000100:00080000:1.0:1776189492.931115:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189492.931123:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189492.931130:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d9eb44b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664356352:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189492.931208:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9143 00010000:02020000:2.0:1776189492.956297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189493.212393:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189493.244870:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189493.471457:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189493.724289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189493.980486:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189494.236747:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189494.270706:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189494.492352:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189494.748220:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189495.006870:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189495.260810:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189495.293029:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189495.516287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189495.772798:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189496.028672:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189496.285205:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189496.316544:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189496.540324:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189496.699427:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189496.699511:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189496.699548:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189496.699643:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654957824 00000100:00100000:0.0:1776189496.699690:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654957760 00000100:00100000:0.0:1776189496.699733:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10040 00000100:00100000:0.0:1776189496.699739:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000895d417a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654957760:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189496.699811:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000895d417a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654957760:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 71us (263us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189496.699818:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10040 00000100:00100000:0.0:1776189496.699837:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57433 00000100:00100000:0.0:1776189496.699841:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f078b3a8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654957824:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776189496.699858:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189496.699885:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f078b3a8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654957824:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 43us (375us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189496.699892:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57433 00000100:00100000:1.0:1776189496.700443:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654957696 00000100:00100000:1.0:1776189496.700527:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1498 00000100:00100000:1.0:1776189496.700532:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000084b636aa pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654957696:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189496.700595:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000084b636aa pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654957696:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 61us (1175us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189496.700603:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1498 00010000:02020000:2.0:1776189496.796978:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189497.021488:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776189497.021513:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d97d7d85 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664356608:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189497.021579:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189497.021669:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664356608 00000100:00100000:3.0:1776189497.021839:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d729b5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664356672:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189497.021860:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6880 00000100:00100000:1.0:1776189497.021867:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cd150e5b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664356608:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189497.021874:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:1.0:1776189497.021894:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189497.021940:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cd150e5b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664356608:12345-0@lo:13:osp-pre-0-0.0 Request processed in 72us (360us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189497.021947:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6880 00000100:00100000:2.0:1776189497.021981:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d97d7d85 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664356608:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189497.025153:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664356672 00000100:00100000:1.0:1776189497.025230:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6881 00000100:00100000:1.0:1776189497.025236:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000412a95e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664356672:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189497.025271:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189497.025343:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000412a95e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664356672:12345-0@lo:13:osp-pre-1-0.0 Request processed in 106us (3471us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189497.025350:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6881 00000100:00100000:3.0:1776189497.027817:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d729b5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664356672:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189497.052408:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189497.308511:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189497.340817:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189497.564274:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189497.820283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189498.044427:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776189498.044451:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d729b5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664356736:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189498.044522:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189498.044660:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664356736 00000100:00100000:1.0:1776189498.044714:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6882 00000100:00100000:1.0:1776189498.044720:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c6f4c7a3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664356736:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189498.044743:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189498.044782:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c6f4c7a3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664356736:12345-0@lo:13:osp-pre-1-1.0 Request processed in 61us (261us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189498.044789:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6882 00000100:00100000:3.0:1776189498.045037:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000056bf1b01 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664356800:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189498.045066:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189498.045095:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664356800 00000100:00100000:3.0:1776189498.045116:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6883 00000100:00100000:3.0:1776189498.045120:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d9eb44b4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664356800:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776189498.045137:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189498.045165:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d9eb44b4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664356800:12345-0@lo:13:osp-pre-0-1.0 Request processed in 44us (99us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189498.045170:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6883 00000100:00100000:3.0:1776189498.045224:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000056bf1b01 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664356800:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189498.045322:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d729b5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664356736:0@lo:13:osp-pre-1-1.0 00000100:00080000:2.0:1776189498.045436:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189498.045441:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189498.045445:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189498.045449:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9639 00000100:00080000:2.0:1776189498.045452:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00080000:2.0:1776189498.045456:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189498.045474:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001b0d11fc x1862459664356992/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189498.045492:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189498.045495:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189498.045496:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189498.045499:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9639 00000100:00080000:2.0:1776189498.045500:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:2.0:1776189498.045503:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189498.045512:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000d3414bd x1862459664357120/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189498.045522:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189498.045524:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189498.045525:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189498.045528:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9639 00000100:00080000:2.0:1776189498.045531:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 11s 00000100:00080000:2.0:1776189498.045533:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189498.045542:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f6d90a77 x1862459664357184/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189498.045619:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c731e56f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664356864:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189498.045652:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189498.045683:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009f0944ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664357312:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189498.045705:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189498.045723:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776189498.045727:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002dce0820 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664357248:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189498.045747:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189498.045772:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664357312 00000100:00100000:1.0:1776189498.045795:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664357248 00000100:00100000:1.0:1776189498.045812:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8619 00000100:00100000:1.0:1776189498.045816:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007df90c3e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664357312:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189498.045851:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007df90c3e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664357312:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (145us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189498.045857:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8619 00000100:00100000:1.0:1776189498.045869:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8620 00000100:00100000:1.0:1776189498.045872:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d97d7d85 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664357248:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189498.045897:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d97d7d85 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664357248:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (150us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189498.045902:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8620 00000100:00100000:2.0:1776189498.046035:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664356864 00000100:00100000:2.0:1776189498.046060:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1499 00000100:00100000:2.0:1776189498.046064:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c1f77e46 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664356864:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189498.046094:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c1f77e46 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664356864:12345-0@lo:400:kworker/u8:0.0 Request processed in 30us (442us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189498.046100:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1499 00000100:00100000:2.0:1776189498.046142:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001a12eabb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664357056:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189498.046167:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189498.046221:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:2.0:1776189498.046225:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000880386e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664356928:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189498.046246:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189498.046257:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000061a1fd5c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664357376:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189498.046275:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189498.046298:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664357056 00000100:00100000:2.0:1776189498.046319:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664356928 00000100:00100000:2.0:1776189498.046333:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664357376 00000100:00100000:2.0:1776189498.046347:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10041 00000100:00100000:2.0:1776189498.046351:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004490682c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664357056:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189498.046379:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004490682c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664357056:12345-0@lo:400:kworker/u8:0.0 Request processed in 27us (211us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189498.046384:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10041 00000100:00100000:2.0:1776189498.046394:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10042 00000100:00100000:2.0:1776189498.046398:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006145bcbc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664356928:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189498.046421:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006145bcbc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664356928:12345-0@lo:400:kworker/u8:0.0 Request processed in 22us (175us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189498.046426:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10042 00000100:00100000:2.0:1776189498.046436:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10043 00000100:00100000:2.0:1776189498.046438:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bbf4e716 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664357376:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189498.046460:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bbf4e716 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664357376:12345-0@lo:400:kworker/u8:0.0 Request processed in 21us (185us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189498.046465:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10043 00000100:00100000:2.0:1776189498.046513:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000880386e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664356928:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189498.046521:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000061a1fd5c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664357376:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189498.046530:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001a12eabb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664357056:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189498.046640:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001b0d11fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664356992:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189498.046664:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189498.046694:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d3414bd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664357120:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189498.046715:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189498.046728:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f6d90a77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664357184:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189498.046748:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189498.046773:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664357184 00000100:00100000:2.0:1776189498.046790:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9144 00000100:00100000:2.0:1776189498.046793:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001fc79665 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664357184:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189498.046807:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189498.046837:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001fc79665 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664357184:12345-0@lo:38:kworker/u8:0.0 Request processed in 42us (88us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189498.046842:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9144 00000100:00100000:2.0:1776189498.046856:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664356992 00000100:00100000:2.0:1776189498.046868:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664357120 00000100:00100000:2.0:1776189498.046879:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57434 00000100:00100000:2.0:1776189498.046882:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0fdb037 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664356992:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189498.046888:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189498.046909:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0fdb037 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664356992:12345-0@lo:38:kworker/u8:0.0 Request processed in 26us (246us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189498.046915:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57434 00000100:00100000:2.0:1776189498.046924:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57435 00000100:00100000:2.0:1776189498.046927:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000250cea25 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664357120:12345-0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189498.047111:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002dce0820 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664357248:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189498.047128:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c731e56f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664356864:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189498.047137:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009f0944ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664357312:0@lo:400:kworker/u8:0.0 00000100:00080000:0.0:1776189498.047303:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189498.047307:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189498.047312:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001b0d11fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664356992:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776189498.047320:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189498.047323:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189498.047325:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f6d90a77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664357184:0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189498.050804:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189498.050897:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000250cea25 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664357120:12345-0@lo:38:kworker/u8:0.0 Request processed in 3968us (4181us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189498.050909:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57435 00000100:00080000:0.0:1776189498.055371:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189498.055383:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189498.055391:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d3414bd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664357120:0@lo:38:kworker/u8:0.0 00010000:02020000:2.0:1776189498.077650:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189498.332498:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189498.364567:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189498.590951:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189498.844755:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189499.100229:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189499.358707:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189499.388993:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189499.612288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189499.868284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189500.124907:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189500.381982:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189500.412488:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189500.636313:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189500.892301:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189501.148320:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189501.404311:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189501.436382:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189501.660289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189501.822626:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189501.822681:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189501.822711:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654957952 00000100:00100000:0.0:1776189501.822786:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10044 00000100:00100000:0.0:1776189501.822801:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000074a40efb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654957952:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1776189501.822801:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189501.822864:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654958016 00000100:00100000:3.0:1776189501.822890:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57436 00000100:00100000:3.0:1776189501.822895:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000011679d4d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654958016:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00000100:00100000:0.0:1776189501.822902:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000074a40efb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654957952:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 105us (285us total) trans 0 rc 0/0 00010000:02000000:3.0:1776189501.822914:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189501.822915:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10044 00000100:00100000:3.0:1776189501.822959:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000011679d4d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654958016:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 62us (165us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189501.822967:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57436 00000100:00100000:2.0:1776189501.823201:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654957888 00000100:00100000:2.0:1776189501.823249:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1500 00000100:00100000:2.0:1776189501.823254:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f055c03f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654957888:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776189501.823298:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f055c03f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654957888:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 43us (618us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189501.823304:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1500 00010000:02020000:2.0:1776189501.916284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189502.140493:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776189502.140974:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f09d90e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664357440:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189502.140993:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3185a52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664357504:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189502.144995:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189502.145082:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189502.145677:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664357440 00000100:00100000:2.0:1776189502.146004:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664357504 00000100:00100000:2.0:1776189502.146111:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6884 00000100:00100000:2.0:1776189502.146118:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000068c80116 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664357440:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189502.146154:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189502.146247:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000068c80116 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664357440:12345-0@lo:13:osp-pre-1-0.0 Request processed in 129us (1596us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189502.146254:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6884 00000100:00100000:2.0:1776189502.146271:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6885 00000100:00100000:2.0:1776189502.146275:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e42142fc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664357504:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189502.146291:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189502.146345:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e42142fc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664357504:12345-0@lo:13:osp-pre-0-0.0 Request processed in 68us (1263us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189502.146351:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6885 00000100:00100000:0.0:1776189502.146567:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f09d90e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664357440:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189502.146622:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3185a52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664357504:0@lo:13:osp-pre-0-0.0 00010000:02020000:2.0:1776189502.172280:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189502.428604:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189502.461627:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189502.684335:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189502.940294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189503.164742:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5f1e324 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664357568:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189503.164818:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1776189503.164904:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189503.164910:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189503.164913:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189503.164918:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9645 00000100:00080000:2.0:1776189503.164922:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:2.0:1776189503.164927:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189503.164965:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b4a5ce06 x1862459664357696/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189503.164988:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189503.164991:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189503.164992:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189503.164996:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9645 00000100:00080000:2.0:1776189503.164998:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00080000:2.0:1776189503.165001:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189503.165012:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000602ee85e x1862459664357824/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189503.165019:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189503.165022:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189503.165023:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189503.165026:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9645 00000100:00080000:2.0:1776189503.165027:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00080000:2.0:1776189503.165030:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189503.165039:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004be216e7 x1862459664357888/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189503.165112:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664357568 00000100:00100000:3.0:1776189503.165159:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1501 00000100:00100000:3.0:1776189503.165164:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000353f317b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664357568:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189503.165724:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e071bb1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664358016:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189503.165759:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189503.166692:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000075d6a9fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664357760:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189503.166741:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189503.166764:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000058c450e1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664358144:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189503.166785:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189503.166807:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:2.0:1776189503.166812:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000576bf6bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664357632:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189503.166834:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189503.166850:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000154c017b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664358080:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189503.166875:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189503.166909:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664357760 00000100:00100000:2.0:1776189503.166960:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664358144 00000100:00100000:2.0:1776189503.166979:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664357632 00000100:00100000:2.0:1776189503.166995:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10045 00000100:00100000:2.0:1776189503.167000:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001929a2c4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664357760:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189503.167044:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001929a2c4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664357760:12345-0@lo:400:kworker/u8:0.0 Request processed in 43us (304us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189503.167052:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10045 00000100:00100000:2.0:1776189503.167069:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10046 00000100:00100000:2.0:1776189503.167072:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000d3414bd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664358144:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189503.171580:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000d3414bd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664358144:12345-0@lo:400:kworker/u8:0.0 Request processed in 4484us (4773us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189503.171599:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10046 00000100:00100000:2.0:1776189503.171635:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10047 00000100:00100000:2.0:1776189503.171642:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001a12eabb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664357632:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189503.171714:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001a12eabb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664357632:12345-0@lo:400:kworker/u8:0.0 Request processed in 73us (4880us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189503.171720:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10047 00000100:00100000:2.0:1776189503.171752:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664358080 00000100:00100000:2.0:1776189503.171798:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8621 00000100:00100000:2.0:1776189503.171802:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f6d90a77 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664358080:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189503.171850:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f6d90a77 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664358080:12345-0@lo:400:kworker/u8:0.0 Request processed in 47us (4975us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189503.171856:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8621 00000100:00100000:2.0:1776189503.171910:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000576bf6bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664357632:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189503.171923:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000154c017b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664358080:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189503.171934:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000075d6a9fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664357760:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189503.171943:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000058c450e1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664358144:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189503.172016:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664358016 00000100:00100000:2.0:1776189503.172051:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6886 00000100:00100000:2.0:1776189503.172054:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c3185a52 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664358016:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189503.172070:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189503.178522:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c3185a52 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664358016:12345-0@lo:13:osp-pre-0-1.0 Request processed in 6456us (12752us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189503.178539:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6886 00000100:00100000:2.0:1776189503.178737:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4a5ce06 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664357696:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189503.178790:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189503.178809:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602ee85e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664357824:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189503.178828:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189503.178839:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004be216e7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664357888:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189503.178859:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189503.178901:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664357696 00000100:00100000:2.0:1776189503.178927:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664357824 00000100:00100000:2.0:1776189503.178942:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57437 00000100:00100000:2.0:1776189503.178963:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002dce0820 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664357696:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189503.178977:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189503.179015:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002dce0820 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664357696:12345-0@lo:38:kworker/u8:0.0 Request processed in 51us (225us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189503.179021:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57437 00000100:00100000:2.0:1776189503.179032:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57438 00000100:00100000:2.0:1776189503.179035:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000061a1fd5c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664357824:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189503.179042:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189503.179064:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000061a1fd5c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664357824:12345-0@lo:38:kworker/u8:0.0 Request processed in 28us (236us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189503.179070:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57438 00000100:00080000:2.0:1776189503.179123:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189503.179128:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189503.179132:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4a5ce06 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664357696:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776189503.179141:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189503.179143:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189503.179146:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602ee85e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664357824:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189503.179209:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664357888 00000100:00100000:2.0:1776189503.179227:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9145 00000100:00100000:2.0:1776189503.179230:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000058c450e1 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664357888:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189503.179238:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189503.179280:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000058c450e1 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664357888:12345-0@lo:38:kworker/u8:0.0 Request processed in 47us (419us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189503.179285:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9145 00000100:00080000:2.0:1776189503.179309:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189503.179313:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189503.179315:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004be216e7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664357888:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189503.179351:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bd5316a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664357952:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189503.179377:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189503.179394:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000549bc579 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664358208:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189503.179413:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189503.179430:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776189503.179434:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009f0944ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664358272:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189503.179454:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189503.179477:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664357952 00000100:00100000:2.0:1776189503.179504:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664358208 00000100:00100000:2.0:1776189503.179524:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8622 00000100:00100000:2.0:1776189503.179528:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004be216e7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664357952:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189503.179580:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004be216e7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664357952:12345-0@lo:400:kworker/u8:0.0 Request processed in 51us (202us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189503.179587:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8622 00000100:00100000:2.0:1776189503.179600:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8623 00000100:00100000:2.0:1776189503.179604:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000602ee85e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664358208:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189503.179631:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000602ee85e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664358208:12345-0@lo:400:kworker/u8:0.0 Request processed in 26us (218us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189503.179637:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8623 00000100:00100000:2.0:1776189503.179653:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664358272 00000100:00100000:2.0:1776189503.179675:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6887 00000100:00100000:2.0:1776189503.179679:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b4a5ce06 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664358272:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189503.179695:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189503.179725:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b4a5ce06 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664358272:12345-0@lo:13:osp-pre-1-1.0 Request processed in 45us (270us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189503.179731:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6887 00000100:00100000:2.0:1776189503.179764:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009f0944ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664358272:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189503.179773:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bd5316a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664357952:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189503.179780:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000549bc579 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664358208:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189503.182497:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000353f317b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664357568:12345-0@lo:400:kworker/u8:0.0 Request processed in 17324us (17672us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189503.182513:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1501 00000100:00100000:1.0:1776189503.184799:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5f1e324 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664357568:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189503.184860:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e071bb1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664358016:0@lo:13:osp-pre-0-1.0 00010000:02020000:2.0:1776189503.196295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189503.452247:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189503.484338:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189503.708390:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189503.964315:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189504.220280:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189504.476589:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189504.510204:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189504.732291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189504.988371:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189505.244433:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189505.500284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189505.532506:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189505.756301:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189506.012353:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189506.269885:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189506.524278:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189506.556405:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189506.780282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189506.938076:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189506.938127:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189506.938237:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189506.938420:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654958144 00000100:00100000:1.0:1776189506.938425:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654958208 00000100:00100000:1.0:1776189506.938478:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10048 00000100:00100000:1.0:1776189506.938493:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006235ac3c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654958144:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189506.938518:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654958080 00000100:00100000:0.0:1776189506.938554:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1502 00000100:00100000:0.0:1776189506.938559:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003c5b81e6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654958080:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189506.938577:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006235ac3c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654958144:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 83us (450us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189506.938586:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10048 00000100:00100000:1.0:1776189506.938605:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57439 00000100:00100000:1.0:1776189506.938609:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002ed2a919 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654958208:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776189506.938623:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189506.938758:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002ed2a919 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654958208:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 147us (522us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189506.938765:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57439 00000100:00100000:0.0:1776189506.940312:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003c5b81e6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654958080:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 1748us (2239us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189506.940324:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1502 00010000:02020000:2.0:1776189507.036281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189507.261376:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776189507.261399:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e5117e34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664358336:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189507.261466:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189507.261571:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664358336 00000100:00100000:1.0:1776189507.261621:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6888 00000100:00100000:1.0:1776189507.261627:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e9c407a2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664358336:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189507.261657:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189507.261701:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e9c407a2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664358336:12345-0@lo:13:osp-pre-0-0.0 Request processed in 73us (236us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189507.261709:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6888 00000100:00100000:2.0:1776189507.261878:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776189507.261898:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f61bb34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664358400:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189507.261926:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189507.261976:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664358400 00000100:00100000:1.0:1776189507.262011:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6889 00000100:00100000:1.0:1776189507.262015:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000549bc579 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664358400:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189507.262034:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189507.262077:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000549bc579 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664358400:12345-0@lo:13:osp-pre-1-0.0 Request processed in 61us (150us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189507.262083:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6889 00000100:00100000:0.0:1776189507.262149:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f61bb34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664358400:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189507.262968:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e5117e34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664358336:0@lo:13:osp-pre-0-0.0 00010000:02020000:2.0:1776189507.292313:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189507.549925:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189507.580392:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189507.804586:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189508.060286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189508.284689:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776189508.284714:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f61bb34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664358464:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189508.284792:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189508.284839:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664358464 00000100:00100000:1.0:1776189508.284882:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6890 00000100:00100000:1.0:1776189508.284888:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d6ea5fef pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664358464:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189508.284909:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189508.284948:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d6ea5fef pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664358464:12345-0@lo:13:osp-pre-0-1.0 Request processed in 59us (157us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189508.284955:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6890 00000100:00100000:1.0:1776189508.284997:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f61bb34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664358464:0@lo:13:osp-pre-0-1.0 00000100:00080000:2.0:1776189508.285518:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189508.285523:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189508.285526:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189508.285530:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9650 00000100:00080000:2.0:1776189508.285533:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776189508.285537:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189508.285553:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000154c017b x1862459664358656/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189508.285572:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189508.285575:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189508.285576:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189508.285579:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9650 00000100:00080000:2.0:1776189508.285581:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776189508.285585:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189508.285594:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b5f1e324 x1862459664358784/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189508.285601:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189508.285603:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189508.285604:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189508.285607:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9650 00000100:00080000:2.0:1776189508.285609:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776189508.285611:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189508.285620:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009f0944ec x1862459664358848/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189508.285658:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001b0d11fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664358912:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189508.285690:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189508.285706:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1776189508.285711:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bd5316a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664358528:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189508.285734:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189508.285747:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c731e56f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664358976:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189508.285763:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189508.285781:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000154c017b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664358656:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189508.285801:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189508.285811:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5f1e324 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664358784:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189508.285826:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189508.285836:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009f0944ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664358848:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189508.285852:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189508.285875:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664358912 00000100:00100000:2.0:1776189508.285900:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664358976 00000100:00100000:2.0:1776189508.285913:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8624 00000100:00100000:2.0:1776189508.285916:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000069c2fab0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664358912:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189508.285947:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000069c2fab0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664358912:12345-0@lo:400:kworker/u8:0.0 Request processed in 30us (257us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189508.285952:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8624 00000100:00100000:2.0:1776189508.285963:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8625 00000100:00100000:2.0:1776189508.285966:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000fce6caf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664358976:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189508.285985:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000fce6caf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664358976:12345-0@lo:400:kworker/u8:0.0 Request processed in 19us (223us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189508.285990:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8625 00000100:00100000:2.0:1776189508.286005:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664358656 00000100:00100000:2.0:1776189508.286021:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664358784 00000100:00100000:2.0:1776189508.286031:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57440 00000100:00100000:2.0:1776189508.286034:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d63ca2bb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664358656:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189508.286045:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189508.286072:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d63ca2bb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664358656:12345-0@lo:38:kworker/u8:0.0 Request processed in 37us (271us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189508.286077:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57440 00000100:00100000:2.0:1776189508.286086:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57441 00000100:00100000:2.0:1776189508.286089:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001f927d70 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664358784:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189508.286094:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189508.286110:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001f927d70 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664358784:12345-0@lo:38:kworker/u8:0.0 Request processed in 20us (284us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189508.286115:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57441 00000100:00100000:2.0:1776189508.286130:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664358848 00000100:00100000:2.0:1776189508.286144:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9146 00000100:00100000:2.0:1776189508.286147:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000df4082ad pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664358848:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189508.286153:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189508.286193:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000df4082ad pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664358848:12345-0@lo:38:kworker/u8:0.0 Request processed in 24us (319us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189508.286199:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9146 00000100:00100000:2.0:1776189508.286224:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664358528 00000100:00100000:2.0:1776189508.286244:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1503 00000100:00100000:2.0:1776189508.286248:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008e1954f2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664358528:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189508.286271:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008e1954f2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664358528:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (538us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189508.286277:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1503 00000100:00080000:2.0:1776189508.286312:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189508.286315:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189508.286318:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000154c017b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664358656:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776189508.286325:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189508.286327:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189508.286330:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5f1e324 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664358784:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776189508.286335:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189508.286337:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189508.286339:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009f0944ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664358848:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189508.286363:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bd5316a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664358528:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189508.286371:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c731e56f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664358976:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189508.286378:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001b0d11fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664358912:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189508.286396:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000075d6a9fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664358592:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189508.286418:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189508.286430:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000880386e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664359040:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189508.286447:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189508.286476:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1776189508.286482:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000576bf6bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664358720:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189508.286500:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189508.286512:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005788f021 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664359104:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189508.286529:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189508.286549:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664358592 00000100:00100000:2.0:1776189508.286570:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664359040 00000100:00100000:3.0:1776189508.287012:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e0bab4ab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664359168:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189508.287041:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189508.287071:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664359168 00000100:00100000:3.0:1776189508.287093:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6891 00000100:00100000:3.0:1776189508.287097:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001706ae06 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664359168:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189508.287112:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189508.287143:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001706ae06 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664359168:12345-0@lo:13:osp-pre-1-1.0 Request processed in 45us (101us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189508.287148:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6891 00000100:00100000:3.0:1776189508.287197:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e0bab4ab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664359168:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189508.287287:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664358720 00000100:00100000:3.0:1776189508.287310:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10049 00000100:00100000:3.0:1776189508.287314:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cc75cc81 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664358592:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189508.287349:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cc75cc81 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664358592:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (931us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189508.287355:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10049 00000100:00100000:3.0:1776189508.287366:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10050 00000100:00100000:3.0:1776189508.287369:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005ff20648 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664358720:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189508.287397:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005ff20648 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664358720:12345-0@lo:400:kworker/u8:0.0 Request processed in 26us (896us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189508.287402:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10050 00000100:00100000:3.0:1776189508.287469:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664359104 00000100:00100000:3.0:1776189508.287490:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8626 00000100:00100000:3.0:1776189508.287494:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000db568034 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664359104:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189508.287522:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000db568034 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664359104:12345-0@lo:400:kworker/u8:0.0 Request processed in 27us (992us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189508.287528:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8626 00000100:00100000:3.0:1776189508.287560:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000576bf6bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664358720:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189508.287570:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005788f021 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664359104:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189508.287578:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000075d6a9fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664358592:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189508.288397:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10051 00000100:00100000:2.0:1776189508.288402:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000694b7c7e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664359040:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189508.288470:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000694b7c7e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664359040:12345-0@lo:400:kworker/u8:0.0 Request processed in 66us (2023us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189508.288476:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10051 00000100:00100000:3.0:1776189508.288532:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000880386e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664359040:0@lo:400:kworker/u8:0.0 00010000:02020000:2.0:1776189508.316296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189508.572287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189508.604357:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189508.828981:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189509.084325:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189509.340286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189509.596408:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189509.628391:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189509.852282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189510.108315:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189510.365150:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189510.620305:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189510.653691:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189510.877201:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189511.135216:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189511.388288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189511.644422:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189511.676402:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189511.901549:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189512.061809:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189512.062219:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189512.062337:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189512.062922:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654958336 00000100:00100000:0.0:1776189512.063045:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654958400 00000100:00100000:0.0:1776189512.063129:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10052 00000100:00100000:0.0:1776189512.063316:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e009ccc0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654958336:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1776189512.063445:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e009ccc0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654958336:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 130us (1227us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189512.063453:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10052 00000100:00100000:0.0:1776189512.063472:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57442 00000100:00100000:0.0:1776189512.063476:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000000115796 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654958400:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:0.0:1776189512.063491:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189512.063515:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000000115796 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654958400:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 39us (1180us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189512.063521:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57442 00000100:00100000:3.0:1776189512.065001:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654958272 00000100:00100000:3.0:1776189512.065093:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1504 00000100:00100000:3.0:1776189512.065099:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001aaffad0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654958272:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1776189512.065227:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001aaffad0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654958272:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 127us (3448us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189512.065235:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1504 00010000:02020000:2.0:1776189512.158474:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189512.380447:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776189512.380464:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776189512.380469:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000071c1b92f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664359232:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189512.380478:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8bdd15 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664359296:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189512.382476:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189512.383630:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664359232 00000100:00100000:2.0:1776189512.383735:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6892 00000100:00100000:2.0:1776189512.383743:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e0bab4ab pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664359232:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189512.383777:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189512.383783:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189512.383869:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e0bab4ab pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664359232:12345-0@lo:13:osp-pre-1-0.0 Request processed in 125us (1398us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189512.383877:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6892 00000100:00100000:0.0:1776189512.384025:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000071c1b92f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664359232:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189512.385441:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664359296 00000100:00100000:3.0:1776189512.385585:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6893 00000100:00100000:3.0:1776189512.385591:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000000005173 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664359296:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189512.385619:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189512.385674:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000000005173 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664359296:12345-0@lo:13:osp-pre-0-0.0 Request processed in 83us (1897us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189512.385681:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6893 00000100:00100000:1.0:1776189512.385723:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8bdd15 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664359296:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189512.413930:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189512.669045:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189512.700576:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189512.924309:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189513.180291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:1.0:1776189513.404417:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189513.404425:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189513.404429:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1776189513.404430:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000733a9e44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664359424:0@lo:13:osp-pre-1-1.0 00000100:00080000:1.0:1776189513.404434:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9655 00000100:00080000:1.0:1776189513.404447:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189513.404453:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189513.404470:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000021503dd8 x1862459664359552/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189513.404490:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189513.404493:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189513.404494:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1776189513.404494:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776189513.404498:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9655 00000100:00080000:1.0:1776189513.404499:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189513.404502:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189513.404509:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008c0d0294 x1862459664359680/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189513.404516:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189513.404518:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189513.404519:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189513.404521:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9655 00000100:00100000:3.0:1776189513.404522:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00080000:1.0:1776189513.404523:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189513.404526:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189513.404527:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ab3412dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664359488:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189513.404536:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000071348c99 x1862459664359744/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189513.404548:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189513.404565:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004d99e80b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664359872:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189513.404587:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189513.404588:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a93ab5a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664359616:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189513.404602:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776189513.404606:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e5117e34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664359936:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189513.404613:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189513.404644:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000067c9d69b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664360000:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189513.404663:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8bdd15 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664359360:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189513.404664:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189513.404694:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189513.404708:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004eaf09e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664359808:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189513.404729:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189513.404761:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664359360 00000100:00100000:2.0:1776189513.404797:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1505 00000100:00100000:2.0:1776189513.404802:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000439c1468 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664359360:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189513.404844:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000439c1468 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664359360:12345-0@lo:400:kworker/u8:1.0 Request processed in 41us (149us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189513.404851:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1505 00000100:00100000:2.0:1776189513.404888:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8bdd15 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664359360:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189513.404906:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664359424 00000100:00100000:2.0:1776189513.404926:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6894 00000100:00100000:2.0:1776189513.404930:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f09d90e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664359424:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189513.404943:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189513.404966:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f09d90e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664359424:12345-0@lo:13:osp-pre-1-1.0 Request processed in 36us (474us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189513.404972:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6894 00000100:00100000:2.0:1776189513.404996:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664359872 00000100:00100000:2.0:1776189513.405015:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664360000 00000100:00100000:2.0:1776189513.405028:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664359808 00000100:00100000:2.0:1776189513.405041:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8627 00000100:00100000:2.0:1776189513.405044:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005788f021 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664359872:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189513.405074:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005788f021 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664359872:12345-0@lo:400:kworker/u8:1.0 Request processed in 30us (487us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189513.405080:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8627 00000100:00100000:2.0:1776189513.405089:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8628 00000100:00100000:2.0:1776189513.405092:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bf60fbf3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664360000:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189513.405115:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bf60fbf3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664360000:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (451us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189513.405120:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8628 00000100:00100000:2.0:1776189513.405127:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8629 00000100:00100000:2.0:1776189513.405130:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000002c44196 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664359808:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189513.405152:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000002c44196 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664359808:12345-0@lo:400:kworker/u8:1.0 Request processed in 21us (423us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189513.405157:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8629 00000100:00100000:2.0:1776189513.405205:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004eaf09e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664359808:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189513.405221:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189513.410775:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000071c1b92f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664360064:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189513.411588:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189513.411734:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664360064 00000100:00100000:0.0:1776189513.411777:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6895 00000100:00100000:0.0:1776189513.411783:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e0967c51 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664360064:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776189513.411807:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189513.413607:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e0967c51 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664360064:12345-0@lo:13:osp-pre-0-1.0 Request processed in 1818us (2017us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189513.413782:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6895 00000100:00100000:0.0:1776189513.414604:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000071c1b92f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664360064:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189513.415414:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004d99e80b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664359872:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189513.415438:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000733a9e44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664359424:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189513.415525:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000021503dd8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664359552:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189513.415576:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189513.415616:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664359488 00000100:00100000:3.0:1776189513.415663:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664359616 00000100:00100000:3.0:1776189513.415679:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664359936 00000100:00100000:3.0:1776189513.415893:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10053 00000100:00100000:3.0:1776189513.415901:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d729b5cd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664359488:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189513.415956:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d729b5cd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664359488:12345-0@lo:400:kworker/u8:1.0 Request processed in 55us (11407us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189513.415964:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10053 00000100:00100000:3.0:1776189513.415981:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10054 00000100:00100000:3.0:1776189513.415986:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004ae527b6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664359616:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189513.416013:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004ae527b6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664359616:12345-0@lo:400:kworker/u8:1.0 Request processed in 27us (11400us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189513.416019:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10054 00000100:00100000:3.0:1776189513.416029:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10055 00000100:00100000:3.0:1776189513.416032:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004eaf09e3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664359936:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189513.416055:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004eaf09e3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664359936:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (10834us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189513.416061:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10055 00000100:00100000:3.0:1776189513.416210:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e5117e34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664359936:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189513.416231:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ab3412dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664359488:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189513.416274:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a93ab5a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664359616:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189513.416285:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000067c9d69b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664360000:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189513.433481:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664359552 00000100:00100000:0.0:1776189513.433539:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57443 00000100:00100000:0.0:1776189513.433546:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004d99e80b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664359552:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189513.433564:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189513.433633:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004d99e80b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664359552:12345-0@lo:38:kworker/u8:1.0 Request processed in 86us (18057us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189513.433640:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57443 00000100:00080000:2.0:1776189513.436999:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189513.437013:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189513.437020:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000021503dd8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664359552:0@lo:38:kworker/u8:1.0 00010000:02020000:0.0:1776189513.437045:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189513.438360:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c0d0294 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664359680:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189513.438452:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189513.438493:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000071348c99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664359744:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189513.438520:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189513.438909:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664359744 00000100:00100000:2.0:1776189513.438955:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9147 00000100:00100000:2.0:1776189513.438962:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000074e02040 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664359744:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189513.438979:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189513.439034:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000074e02040 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664359744:12345-0@lo:38:kworker/u8:1.0 Request processed in 71us (513us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189513.439040:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9147 00000100:00100000:1.0:1776189513.439448:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664359680 00000100:00100000:1.0:1776189513.439485:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57444 00000100:00100000:1.0:1776189513.439489:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000b229330 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664359680:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189513.439497:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189513.439529:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000b229330 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664359680:12345-0@lo:38:kworker/u8:1.0 Request processed in 38us (1078us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189513.439535:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57444 00000100:00080000:0.0:1776189513.439764:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189513.439768:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189513.439773:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c0d0294 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664359680:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776189513.439783:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189513.439785:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189513.439787:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000071348c99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664359744:0@lo:38:kworker/u8:1.0 00010000:02020000:0.0:1776189513.694562:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189513.725579:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189513.948283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189514.204301:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189514.460577:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189514.716592:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189514.748508:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189514.973991:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189515.228310:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189515.484306:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189515.743379:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189515.772554:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189515.997224:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189516.252289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189516.508378:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189516.765134:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189516.801714:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189517.022673:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189517.191924:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189517.191990:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189517.192085:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654958464 00000100:00100000:1.0:1776189517.192142:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1506 00000100:00100000:1.0:1776189517.192148:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006339443e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654958464:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189517.192269:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006339443e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654958464:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 111us (270us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189517.192276:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1506 00000100:00100000:3.0:1776189517.192351:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654958528 00000100:00100000:3.0:1776189517.192396:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10056 00000100:00100000:3.0:1776189517.192401:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006c8bdd15 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654958528:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189517.192432:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006c8bdd15 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654958528:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 31us (516us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189517.192438:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10056 00000100:00100000:3.0:1776189517.192746:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189517.192766:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654958592 00000100:00100000:1.0:1776189517.192795:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57445 00000100:00100000:1.0:1776189517.192800:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003bb8357b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654958592:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776189517.192814:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189517.192850:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003bb8357b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654958592:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 49us (106us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189517.192856:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57445 00010000:02020000:0.0:1776189517.276734:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189517.502065:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776189517.502095:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000071c1b92f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664360128:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189517.502194:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189517.503680:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664360128 00000100:00100000:0.0:1776189517.503760:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6896 00000100:00100000:0.0:1776189517.503766:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000071348c99 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664360128:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189517.503802:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189517.503887:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000071348c99 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664360128:12345-0@lo:13:osp-pre-1-0.0 Request processed in 119us (1716us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189517.503895:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6896 00000100:00100000:3.0:1776189517.504120:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000071c1b92f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664360128:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189517.504364:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000526ed979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664360192:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189517.504406:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189517.504486:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664360192 00000100:00100000:2.0:1776189517.504525:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6897 00000100:00100000:2.0:1776189517.504530:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022af59eb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664360192:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189517.504548:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189517.504590:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022af59eb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664360192:12345-0@lo:13:osp-pre-0-0.0 Request processed in 59us (184us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189517.504596:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6897 00000100:00100000:3.0:1776189517.504721:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000526ed979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664360192:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189517.532448:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189517.788431:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189517.821655:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189518.044395:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189518.300288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189518.524673:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f8f9cfc6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664360256:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189518.524760:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189518.524941:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664360256 00000100:00100000:3.0:1776189518.524991:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6898 00000100:00100000:3.0:1776189518.524996:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000068939cd3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664360256:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189518.525018:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189518.525072:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000068939cd3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664360256:12345-0@lo:13:osp-pre-1-1.0 Request processed in 74us (312us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189518.525080:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6898 00000100:00080000:2.0:1776189518.525278:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189518.525283:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189518.525286:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189518.525291:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9660 00000100:00080000:2.0:1776189518.525294:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776189518.525297:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189518.525441:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000021503dd8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664360384:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189518.525472:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189518.525605:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a93ab5a8 x1862459664360448/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189518.525639:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189518.525642:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189518.525644:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189518.525647:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9660 00000100:00080000:2.0:1776189518.525650:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776189518.525652:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189518.525662:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e5117e34 x1862459664360576/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189518.525670:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189518.525672:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189518.525673:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189518.525676:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9660 00000100:00080000:2.0:1776189518.525678:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776189518.525681:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189518.525869:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a93ab5a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664360448:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189518.525896:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189518.525912:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e5117e34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664360576:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189518.525931:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189518.525957:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664360384 00000100:00100000:3.0:1776189518.525986:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664360448 00000100:00100000:3.0:1776189518.526001:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664360576 00000100:00100000:3.0:1776189518.526014:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10057 00000100:00100000:3.0:1776189518.526018:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008e1f2ed1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664360384:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189518.526052:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008e1f2ed1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664360384:12345-0@lo:400:kworker/u8:1.0 Request processed in 32us (580us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189518.526058:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10057 00000100:00100000:3.0:1776189518.526074:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57446 00000100:00100000:3.0:1776189518.526077:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000624e744d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664360448:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189518.526090:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189518.526119:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000624e744d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664360448:12345-0@lo:38:kworker/u8:1.0 Request processed in 41us (223us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189518.526126:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57446 00000100:00100000:3.0:1776189518.526137:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57447 00000100:00100000:3.0:1776189518.526139:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000001d1f947 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664360576:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189518.526146:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189518.526166:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000001d1f947 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664360576:12345-0@lo:38:kworker/u8:1.0 Request processed in 25us (235us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189518.526199:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57447 00000100:00080000:3.0:1776189518.526434:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189518.526438:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189518.526442:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a93ab5a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664360448:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776189518.526452:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189518.526454:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189518.526456:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e5117e34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664360576:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189518.526491:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000021503dd8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664360384:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189518.526500:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776189518.526506:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ab3412dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664360512:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189518.526529:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189518.526553:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664360512 00000100:00100000:3.0:1776189518.526574:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10058 00000100:00100000:3.0:1776189518.526577:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000063864031 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664360512:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189518.526608:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000063864031 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664360512:12345-0@lo:400:kworker/u8:1.0 Request processed in 30us (80us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189518.526614:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10058 00000100:00100000:3.0:1776189518.526642:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ab3412dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664360512:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189518.526887:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000733a9e44 x1862459664360640/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189518.527027:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c0d0294 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664360320:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189518.527061:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189518.527079:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001b0d11fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664360768:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189518.527102:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189518.527119:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776189518.527124:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000880386e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664360704:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189518.527145:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189518.527169:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664360768 00000100:00100000:0.0:1776189518.527209:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664360704 00000100:00100000:0.0:1776189518.527436:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8630 00000100:00100000:0.0:1776189518.527441:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009fddac3a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664360768:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189518.527478:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009fddac3a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664360768:12345-0@lo:400:kworker/u8:1.0 Request processed in 37us (376us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189518.527484:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8630 00000100:00100000:0.0:1776189518.527497:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8631 00000100:00100000:0.0:1776189518.527500:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005ab67562 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664360704:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189518.527525:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005ab67562 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664360704:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (380us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189518.527531:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8631 00000100:00100000:0.0:1776189518.527557:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000880386e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664360704:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189518.527566:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001b0d11fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664360768:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189518.527611:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664360320 00000100:00100000:0.0:1776189518.527634:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1507 00000100:00100000:0.0:1776189518.527637:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a3281d23 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664360320:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189518.527669:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a3281d23 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664360320:12345-0@lo:400:kworker/u8:1.0 Request processed in 31us (608us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189518.527675:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1507 00000100:00100000:3.0:1776189518.528532:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000733a9e44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664360640:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189518.528574:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189518.528618:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c731e56f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664360832:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189518.528644:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189518.528662:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776189518.528667:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009f0944ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664360896:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189518.528696:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189518.528816:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000075d6a9fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664360960:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189518.528879:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664360640 00000100:00100000:2.0:1776189518.528907:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9148 00000100:00100000:2.0:1776189518.528912:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000041c094ed pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664360640:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189518.528925:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189518.529184:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c0d0294 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664360320:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189518.529245:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664360832 00000100:00100000:0.0:1776189518.529274:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10059 00000100:00100000:0.0:1776189518.529278:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bec72d87 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664360832:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189518.535987:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189518.536073:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f8f9cfc6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664360256:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189518.536157:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664360960 00000100:00100000:2.0:1776189518.543293:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000041c094ed pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664360640:12345-0@lo:38:kworker/u8:1.0 Request processed in 14372us (14711us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189518.543319:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9148 00000100:00100000:2.0:1776189518.543373:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664360896 00000100:00100000:2.0:1776189518.543410:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8632 00000100:00100000:2.0:1776189518.543416:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000086fda571 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+6:3361:x1862459664360896:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189518.543474:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000086fda571 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+6:3361:x1862459664360896:12345-0@lo:400:kworker/u8:1.0 Request processed in 57us (14787us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189518.543481:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8632 00000100:00080000:2.0:1776189518.543532:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189518.543537:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189518.543542:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000733a9e44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664360640:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189518.543611:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6899 00000100:00100000:2.0:1776189518.543616:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004926aec5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664360960:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189518.543632:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189518.543675:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004926aec5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664360960:12345-0@lo:13:osp-pre-0-1.0 Request processed in 58us (7695us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189518.543681:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6899 00000100:00100000:2.0:1776189518.543965:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000075d6a9fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664360960:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189518.547956:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009f0944ec pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664360896:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189518.548435:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bec72d87 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664360832:12345-0@lo:400:kworker/u8:1.0 Request processed in 19154us (19789us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189518.548448:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10059 00000100:00100000:3.0:1776189518.548886:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c731e56f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664360832:0@lo:400:kworker/u8:1.0 00010000:02020000:0.0:1776189518.556447:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189518.812445:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189518.844623:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189519.069206:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189519.324617:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189519.581145:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189519.836283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189519.868759:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189520.092762:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189520.348523:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189520.607431:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189520.860326:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189520.892881:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189521.116287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189521.372814:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189521.628686:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189521.884289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189521.916325:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189522.140281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189522.301322:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189522.301391:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189522.301638:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654958720 00000100:00100000:0.0:1776189522.301712:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10060 00000100:00100000:0.0:1776189522.301722:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000045a10d8d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654958720:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189522.301920:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000045a10d8d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654958720:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 198us (528us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189522.301929:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10060 00000100:00100000:1.0:1776189522.302227:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654958656 00000100:00100000:1.0:1776189522.302269:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1508 00000100:00100000:1.0:1776189522.302274:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b7303d3d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654958656:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189522.302307:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b7303d3d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654958656:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 33us (1000us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189522.302314:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1508 00000100:00100000:0.0:1776189522.311860:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189522.312512:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654958784 00000100:00100000:2.0:1776189522.312583:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57448 00000100:00100000:2.0:1776189522.312592:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000046db4947 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654958784:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776189522.312618:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189522.312710:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000046db4947 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654958784:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 117us (861us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189522.312718:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57448 00010000:02020000:0.0:1776189522.396672:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189522.620969:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776189522.621003:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e71929fa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664361024:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189522.621099:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189522.621282:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776189522.621294:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009257ba59 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664361088:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189522.621326:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189522.631805:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664361024 00000100:00100000:1.0:1776189522.634296:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664361088 00000100:00100000:1.0:1776189522.634381:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6900 00000100:00100000:1.0:1776189522.634389:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000075d6a9fc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664361024:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189522.634438:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189522.634563:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000075d6a9fc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664361024:12345-0@lo:13:osp-pre-0-0.0 Request processed in 174us (13465us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189522.634572:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6900 00000100:00100000:1.0:1776189522.634596:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6901 00000100:00100000:1.0:1776189522.634601:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b4a5ce06 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664361088:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189522.634619:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189522.635888:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b4a5ce06 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664361088:12345-0@lo:13:osp-pre-1-0.0 Request processed in 1278us (14555us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189522.635902:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6901 00000100:00100000:2.0:1776189522.636275:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009257ba59 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664361088:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189522.636336:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e71929fa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664361024:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189522.652335:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189522.908411:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189522.941212:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189523.164833:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189523.421538:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:1.0:1776189523.644612:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189523.644619:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189523.644623:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189523.644632:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9665 00000100:00080000:1.0:1776189523.644636:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00080000:1.0:1776189523.644641:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189523.644660:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d92ffa01 x1862459664361344/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189523.644681:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189523.644684:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189523.644685:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189523.644689:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9665 00000100:00080000:1.0:1776189523.644691:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00080000:1.0:1776189523.644695:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189523.644704:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b7a82fb0 x1862459664361472/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189523.644711:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189523.644714:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189523.644715:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189523.644719:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9665 00000100:00080000:1.0:1776189523.644720:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 6s 00000100:00080000:1.0:1776189523.644723:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189523.644733:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000053ba7e65 x1862459664361536/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189523.644854:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000684e24e8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664361408:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189523.644918:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189523.644950:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000526ed979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664361792:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189523.644974:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189523.644998:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:3.0:1776189523.645004:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000549bc579 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664361152:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189523.645027:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189523.645038:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f059006a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664361600:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189523.645059:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189523.645071:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000067c9d69b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664361856:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189523.645098:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189523.645136:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02fb24b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664361280:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189523.645156:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189523.645168:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000068939cd3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664361728:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189523.645216:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189523.645231:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:3.0:1776189523.645237:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009820cd1e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664361216:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189523.645263:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189523.645275:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000022f2aa27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664361664:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189523.645294:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189523.645305:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9e630d1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664361920:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189523.645324:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189523.645353:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664361792 00000100:00100000:3.0:1776189523.645392:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664361600 00000100:00100000:3.0:1776189523.645405:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664361856 00000100:00100000:3.0:1776189523.645420:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664361664 00000100:00100000:3.0:1776189523.645437:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8633 00000100:00100000:3.0:1776189523.645442:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022af59eb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664361792:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189523.645486:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022af59eb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664361792:12345-0@lo:400:kworker/u8:0.0 Request processed in 43us (511us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189523.645493:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8633 00000100:00100000:3.0:1776189523.645509:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8634 00000100:00100000:3.0:1776189523.645513:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009f0944ec pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664361600:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189523.645550:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009f0944ec pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664361600:12345-0@lo:400:kworker/u8:0.0 Request processed in 36us (490us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189523.645555:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8634 00000100:00100000:3.0:1776189523.645564:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8635 00000100:00100000:3.0:1776189523.645567:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c3185a52 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664361856:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189523.645589:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c3185a52 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664361856:12345-0@lo:400:kworker/u8:0.0 Request processed in 21us (491us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189523.645593:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8635 00000100:00100000:3.0:1776189523.645604:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8636 00000100:00100000:3.0:1776189523.645607:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000071348c99 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664361664:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189523.645634:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000071348c99 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664361664:12345-0@lo:400:kworker/u8:0.0 Request processed in 26us (340us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189523.645640:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8636 00000100:00100000:3.0:1776189523.645679:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000022f2aa27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664361664:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189523.645705:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f059006a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664361600:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189523.645713:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000067c9d69b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664361856:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189523.645720:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000526ed979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664361792:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189523.645801:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d92ffa01 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664361344:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189523.645829:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189523.645845:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b7a82fb0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664361472:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189523.645861:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189523.645874:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000053ba7e65 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664361536:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189523.645896:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189523.645931:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664361408 00000100:00100000:2.0:1776189523.645959:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664361280 00000100:00100000:2.0:1776189523.645971:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664361728 00000100:00100000:2.0:1776189523.645979:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664361344 00000100:00100000:2.0:1776189523.645987:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664361472 00000100:00100000:2.0:1776189523.645995:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10061 00000100:00100000:2.0:1776189523.645999:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e71929fa pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664361408:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189523.646030:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e71929fa pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664361408:12345-0@lo:400:kworker/u8:0.0 Request processed in 30us (1112us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189523.646036:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10061 00000100:00100000:2.0:1776189523.646047:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10062 00000100:00100000:2.0:1776189523.646050:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d6ea5fef pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664361280:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189523.646072:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d6ea5fef pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664361280:12345-0@lo:400:kworker/u8:0.0 Request processed in 22us (916us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189523.646078:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10062 00000100:00100000:2.0:1776189523.646086:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10063 00000100:00100000:2.0:1776189523.646089:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000000005173 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664361728:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189523.646108:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000000005173 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664361728:12345-0@lo:400:kworker/u8:0.0 Request processed in 18us (892us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189523.646113:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10063 00000100:00100000:2.0:1776189523.646133:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57449 00000100:00100000:2.0:1776189523.646136:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000576bf6bf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664361344:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189523.646149:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189523.646197:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000576bf6bf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664361344:12345-0@lo:38:kworker/u8:0.0 Request processed in 60us (368us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189523.646202:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57449 00000100:00100000:2.0:1776189523.646213:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57450 00000100:00100000:2.0:1776189523.646215:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bd5316a5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664361472:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189523.646221:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189523.646240:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bd5316a5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664361472:12345-0@lo:38:kworker/u8:0.0 Request processed in 23us (378us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189523.646245:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57450 00000100:00100000:2.0:1776189523.646272:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664361536 00000100:00100000:2.0:1776189523.646288:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9149 00000100:00100000:2.0:1776189523.646290:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b5f1e324 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664361536:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189523.646296:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189523.646317:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b5f1e324 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664361536:12345-0@lo:38:kworker/u8:0.0 Request processed in 25us (421us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189523.646322:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9149 00000100:00080000:2.0:1776189523.646346:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189523.646349:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189523.646352:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d92ffa01 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664361344:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776189523.646360:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189523.646362:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189523.646364:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b7a82fb0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664361472:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776189523.646371:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189523.646373:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189523.646375:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000053ba7e65 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664361536:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189523.646506:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664361216 00000100:00100000:0.0:1776189523.646542:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1509 00000100:00100000:0.0:1776189523.646545:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000089048673 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664361216:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189523.646576:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000089048673 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664361216:12345-0@lo:400:kworker/u8:0.0 Request processed in 30us (1314us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189523.646582:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1509 00000100:00100000:1.0:1776189523.646676:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664361152 00000100:00100000:1.0:1776189523.646838:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664361920 00000100:00100000:1.0:1776189523.646910:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6902 00000100:00100000:1.0:1776189523.646914:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f8f9cfc6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664361152:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189523.646929:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189523.646961:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f8f9cfc6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664361152:12345-0@lo:13:osp-pre-0-1.0 Request processed in 46us (1934us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189523.646967:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6902 00000100:00100000:1.0:1776189523.646983:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6903 00000100:00100000:1.0:1776189523.646986:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c0d0294 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664361920:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189523.646996:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189523.647019:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c0d0294 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664361920:12345-0@lo:13:osp-pre-1-1.0 Request processed in 32us (1695us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189523.647025:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6903 00000100:00100000:3.0:1776189523.648308:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009820cd1e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664361216:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189523.648333:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9e630d1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664361920:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189523.648345:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02fb24b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664361280:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189523.648355:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000068939cd3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664361728:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189523.648410:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000549bc579 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664361152:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189523.648423:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000684e24e8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664361408:0@lo:400:kworker/u8:0.0 00010000:02020000:0.0:1776189523.676315:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189523.932327:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189523.966365:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189524.189691:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189524.444580:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189524.700879:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189524.956937:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189524.991325:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189525.212455:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189525.468411:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189525.726576:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189525.980478:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189526.016867:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189526.236284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189526.493206:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189526.748446:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189527.004281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189527.036684:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189527.260385:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189527.419293:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189527.419348:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189527.419384:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189527.419487:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654958912 00000100:00100000:0.0:1776189527.419564:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654958976 00000100:00100000:0.0:1776189527.419582:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10064 00000100:00100000:0.0:1776189527.419588:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000154c017b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654958912:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1776189527.419667:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000154c017b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654958912:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 77us (318us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189527.419676:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10064 00000100:00100000:1.0:1776189527.419680:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654958848 00000100:00100000:1.0:1776189527.419718:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1510 00000100:00100000:1.0:1776189527.419723:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000007949e4c pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654958848:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776189527.419753:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000007949e4c pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654958848:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 29us (472us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189527.419764:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1510 00000100:00100000:1.0:1776189527.419809:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57451 00000100:00100000:1.0:1776189527.419814:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000549bc579 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654958976:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:1.0:1776189527.419829:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189527.419884:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000549bc579 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654958976:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 70us (500us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189527.419891:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57451 00010000:02020000:0.0:1776189527.516406:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189527.740517:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776189527.740540:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ab3412dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664361984:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189527.740629:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189527.740998:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b34cb071 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664362048:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189527.741026:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189527.741120:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664361984 00000100:00100000:0.0:1776189527.741166:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664362048 00000100:00100000:0.0:1776189527.741222:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6904 00000100:00100000:0.0:1776189527.741228:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000684e24e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664361984:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776189527.741255:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189527.741293:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000684e24e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664361984:12345-0@lo:13:osp-pre-0-0.0 Request processed in 65us (666us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189527.741300:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6904 00000100:00100000:0.0:1776189527.741314:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6905 00000100:00100000:0.0:1776189527.741323:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000068939cd3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664362048:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189527.741338:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189527.741435:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ab3412dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664361984:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189527.747291:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000068939cd3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664362048:12345-0@lo:13:osp-pre-1-0.0 Request processed in 5957us (6256us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189527.747309:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6905 00000100:00100000:1.0:1776189527.752189:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b34cb071 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664362048:0@lo:13:osp-pre-1-0.0 00010000:02020000:0.0:1776189527.772638:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189528.028302:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189528.060423:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189528.285661:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189528.540298:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:3.0:1776189528.766716:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189528.766723:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189528.766726:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189528.766733:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9670 00000100:00080000:3.0:1776189528.766738:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00080000:3.0:1776189528.766742:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189528.766762:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a93ab5a8 x1862459664362240/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189528.766787:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189528.766790:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189528.766791:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189528.766808:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9670 00000100:00080000:3.0:1776189528.766810:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:3.0:1776189528.766813:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189528.766824:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f9e630d1 x1862459664362368/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189528.766830:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189528.766833:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189528.766834:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189528.766836:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9670 00000100:00080000:3.0:1776189528.766838:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 11s 00000100:00080000:3.0:1776189528.766840:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189528.766853:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001a690fbd x1862459664362432/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189528.766900:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ab3412dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664362304:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189528.766958:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189528.766985:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001a975e34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664362688:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189528.767020:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189528.767043:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1776189528.767048:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e5117e34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664362176:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189528.767068:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189528.767080:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005ee0b9e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664362624:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189528.767097:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189528.767126:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664362688 00000100:00100000:3.0:1776189528.767162:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8637 00000100:00100000:3.0:1776189528.767167:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001fe1325f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664362688:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189528.767229:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001fe1325f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664362688:12345-0@lo:400:kworker/u8:0.0 Request processed in 61us (220us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189528.767236:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8637 00000100:00100000:3.0:1776189528.767272:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001a975e34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664362688:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189528.767298:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664362304 00000100:00100000:3.0:1776189528.767319:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664362176 00000100:00100000:3.0:1776189528.767332:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664362624 00000100:00100000:3.0:1776189528.767344:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10065 00000100:00100000:3.0:1776189528.767348:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c510b01d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664362304:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189528.767380:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c510b01d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664362304:12345-0@lo:400:kworker/u8:0.0 Request processed in 32us (423us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189528.767389:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10065 00000100:00100000:3.0:1776189528.767400:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10066 00000100:00100000:3.0:1776189528.767403:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c8b239a6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664362176:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189528.767428:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c8b239a6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664362176:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (359us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189528.767433:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10066 00000100:00100000:1.0:1776189528.767646:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000021503dd8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664362112:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189528.767680:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189528.767699:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000005ed8ead pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664362560:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189528.767722:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189528.767738:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1776189528.767743:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000023a5b7b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664362496:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189528.767763:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189528.767774:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b39ea587 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664362752:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189528.767803:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189528.767830:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664362112 00000100:00100000:1.0:1776189528.767854:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1511 00000100:00100000:1.0:1776189528.767858:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c1af929a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664362112:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189528.767887:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c1af929a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664362112:12345-0@lo:400:kworker/u8:0.0 Request processed in 28us (207us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189528.767892:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1511 00000100:00100000:1.0:1776189528.767916:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664362560 00000100:00100000:1.0:1776189528.767931:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664362496 00000100:00100000:1.0:1776189528.767945:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664362752 00000100:00100000:1.0:1776189528.767962:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8638 00000100:00100000:1.0:1776189528.767965:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d02fb24b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664362560:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189528.767991:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d02fb24b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664362560:12345-0@lo:400:kworker/u8:0.0 Request processed in 25us (270us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189528.767996:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8638 00000100:00100000:1.0:1776189528.768006:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8639 00000100:00100000:1.0:1776189528.768022:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009820cd1e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664362496:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189528.768045:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009820cd1e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664362496:12345-0@lo:400:kworker/u8:0.0 Request processed in 22us (282us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189528.768049:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8639 00000100:00100000:1.0:1776189528.768057:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8640 00000100:00100000:1.0:1776189528.768060:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053ba7e65 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664362752:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189528.768076:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053ba7e65 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664362752:12345-0@lo:400:kworker/u8:0.0 Request processed in 15us (286us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189528.768080:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8640 00000100:00100000:1.0:1776189528.768109:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000023a5b7b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664362496:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189528.768116:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b39ea587 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664362752:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189528.768123:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000021503dd8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664362112:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189528.768129:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000005ed8ead pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664362560:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189528.768157:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10067 00000100:00100000:1.0:1776189528.768160:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003189c953 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664362624:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189528.768201:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003189c953 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664362624:12345-0@lo:400:kworker/u8:0.0 Request processed in 39us (1103us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189528.768206:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10067 00000100:00100000:2.0:1776189528.768300:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a93ab5a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664362240:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189528.768328:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189528.768337:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9e630d1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664362368:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189528.768356:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189528.768364:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001a690fbd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664362432:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189528.768385:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189528.768413:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664362240 00000100:00100000:2.0:1776189528.768426:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664362368 00000100:00100000:2.0:1776189528.768435:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57452 00000100:00100000:2.0:1776189528.768450:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000db0032a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664362240:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189528.768463:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189528.768495:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000db0032a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664362240:12345-0@lo:38:kworker/u8:0.0 Request processed in 45us (167us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189528.768501:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57452 00000100:00100000:2.0:1776189528.768512:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57453 00000100:00100000:2.0:1776189528.768515:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b34cb071 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664362368:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189528.768522:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189528.768540:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b34cb071 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664362368:12345-0@lo:38:kworker/u8:0.0 Request processed in 24us (185us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189528.768545:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57453 00000100:00100000:2.0:1776189528.768581:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664362432 00000100:00100000:2.0:1776189528.768598:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9150 00000100:00100000:2.0:1776189528.768601:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000088ec3431 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664362432:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189528.768608:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189528.768630:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000088ec3431 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664362432:12345-0@lo:38:kworker/u8:0.0 Request processed in 28us (246us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189528.768636:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9150 00000100:00080000:2.0:1776189528.768659:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189528.768663:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189528.768666:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a93ab5a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664362240:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776189528.768672:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189528.768674:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189528.768676:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9e630d1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664362368:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776189528.768681:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189528.768683:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189528.768685:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001a690fbd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664362432:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189528.768743:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e5117e34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664362176:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189528.768754:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005ee0b9e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664362624:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189528.768763:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ab3412dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664362304:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189528.768790:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d3cfd23 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664362816:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189528.768813:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189528.768837:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664362816 00000100:00100000:2.0:1776189528.768858:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6906 00000100:00100000:2.0:1776189528.768861:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ace99b9b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664362816:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189528.768873:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189528.768898:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ace99b9b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664362816:12345-0@lo:13:osp-pre-0-1.0 Request processed in 36us (85us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189528.768903:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6906 00000100:00100000:2.0:1776189528.768922:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d3cfd23 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664362816:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189528.769133:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776189528.769141:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000279e850a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664362880:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189528.769165:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189528.769202:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664362880 00000100:00100000:0.0:1776189528.769222:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6907 00000100:00100000:0.0:1776189528.769225:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e5ef59b0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664362880:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189528.769234:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189528.769257:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e5ef59b0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664362880:12345-0@lo:13:osp-pre-1-1.0 Request processed in 31us (93us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189528.769262:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6907 00000100:00100000:0.0:1776189528.769283:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000279e850a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664362880:0@lo:13:osp-pre-1-1.0 00010000:02020000:0.0:1776189528.796285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189529.052644:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189529.084864:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189529.308309:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189529.564832:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189529.822396:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189530.076286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189530.108415:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189530.332268:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189530.588326:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189530.844325:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189531.100380:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189531.132456:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189531.356330:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189531.612286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189531.868592:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189532.124579:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189532.156398:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189532.380675:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189532.565270:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189532.565324:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189532.565366:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189532.565447:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654959104 00000100:00100000:0.0:1776189532.565499:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654959168 00000100:00100000:0.0:1776189532.565520:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10068 00000100:00100000:0.0:1776189532.565531:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ccb5377d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654959104:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776189532.565608:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57454 00000100:00100000:1.0:1776189532.565614:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000014a0e5e7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654959168:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00000100:00100000:0.0:1776189532.565631:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ccb5377d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654959104:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 99us (369us total) trans 0 rc 0/0 00010000:02000000:1.0:1776189532.565631:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189532.565639:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10068 00000100:00100000:1.0:1776189532.565662:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000014a0e5e7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654959168:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 48us (297us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189532.565668:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57454 00000100:00100000:3.0:1776189532.566198:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654959040 00000100:00100000:3.0:1776189532.566247:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1512 00000100:00100000:3.0:1776189532.566253:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e2219b01 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654959040:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1776189532.566298:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e2219b01 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654959040:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 45us (975us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189532.566305:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1512 00010000:02020000:0.0:1776189532.636559:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189532.860499:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776189532.860522:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001a975e34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664362944:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189532.860587:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189532.861087:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664362944 00000100:00100000:1.0:1776189532.861137:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6908 00000100:00100000:1.0:1776189532.861143:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000099b8eb0d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664362944:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189532.861192:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189532.861258:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000099b8eb0d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664362944:12345-0@lo:13:osp-pre-0-0.0 Request processed in 113us (671us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189532.861266:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6908 00000100:00100000:0.0:1776189532.861456:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000403a47d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664363008:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189532.861488:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189532.861559:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664363008 00000100:00100000:2.0:1776189532.861589:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6909 00000100:00100000:2.0:1776189532.861593:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000279e850a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664363008:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189532.861610:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189532.861640:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000279e850a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664363008:12345-0@lo:13:osp-pre-1-0.0 Request processed in 46us (152us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189532.861650:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6909 00000100:00100000:0.0:1776189532.861724:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000403a47d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664363008:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189532.861808:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001a975e34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664362944:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189532.892340:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189533.148301:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189533.180496:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189533.404353:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189533.662127:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:3.0:1776189533.884362:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189533.884369:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189533.884372:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189533.884383:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9675 00000100:00080000:3.0:1776189533.884388:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1776189533.884395:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189533.884418:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f34aeb18 x1862459664363264/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189533.884426:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004ac5ba05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664363136:0@lo:400:kworker/u8:1.0 00000100:00080000:3.0:1776189533.884450:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189533.884454:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189533.884457:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189533.884460:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9675 00000100:00080000:3.0:1776189533.884462:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1776189533.884466:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189533.884501:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189533.884617:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005ee0b9e5 x1862459664363392/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189533.884625:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189533.884628:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189533.884629:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189533.884633:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9675 00000100:00100000:2.0:1776189533.884634:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00080000:3.0:1776189533.884636:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1776189533.884641:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189533.884647:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004d2de401 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664363200:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189533.884654:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001a690fbd x1862459664363456/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189533.884692:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189533.884723:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b39ea587 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664363584:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189533.884740:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f34aeb18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664363264:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189533.884758:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001a975e34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664363072:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189533.884760:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189533.884780:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189533.884789:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776189533.884808:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000023a5b7b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664363648:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189533.884887:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000048c7c2e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664363328:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189533.885200:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005ee0b9e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664363392:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189533.885225:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189533.885238:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001a690fbd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664363456:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189533.885258:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189533.885306:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664363584 00000100:00100000:3.0:1776189533.885352:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8641 00000100:00100000:3.0:1776189533.885357:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000201d0f3c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664363584:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189533.885397:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000201d0f3c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664363584:12345-0@lo:400:kworker/u8:1.0 Request processed in 40us (641us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189533.885408:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8641 00000100:00100000:3.0:1776189533.885429:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664363136 00000100:00100000:3.0:1776189533.885450:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664363264 00000100:00100000:3.0:1776189533.885461:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664363392 00000100:00100000:3.0:1776189533.885887:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10069 00000100:00100000:3.0:1776189533.885892:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cbabaed7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664363136:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189533.885938:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cbabaed7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664363136:12345-0@lo:400:kworker/u8:1.0 Request processed in 45us (1439us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189533.885944:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10069 00000100:00100000:3.0:1776189533.885955:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57455 00000100:00100000:3.0:1776189533.885958:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f4f823f8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664363264:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189533.885972:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189533.886005:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f4f823f8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664363264:12345-0@lo:38:kworker/u8:1.0 Request processed in 45us (1227us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189533.886010:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57455 00000100:00100000:3.0:1776189533.886020:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57456 00000100:00100000:3.0:1776189533.886023:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c06060ec pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664363392:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189533.886030:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189533.886051:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c06060ec pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664363392:12345-0@lo:38:kworker/u8:1.0 Request processed in 27us (827us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189533.886056:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57456 00000100:00080000:3.0:1776189533.886661:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189533.886666:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189533.886671:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f34aeb18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664363264:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776189533.886685:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189533.886688:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189533.886690:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005ee0b9e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664363392:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189533.886747:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664363456 00000100:00100000:3.0:1776189533.886774:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9151 00000100:00100000:3.0:1776189533.886778:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000b36a9c1 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664363456:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189533.886786:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189533.886822:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000b36a9c1 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664363456:12345-0@lo:38:kworker/u8:1.0 Request processed in 44us (1563us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189533.886828:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9151 00000100:00080000:3.0:1776189533.886859:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189533.886862:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189533.886865:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001a690fbd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664363456:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189533.886961:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776189533.886970:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002e9e0e35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664363840:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189533.887000:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189533.887032:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664363200 00000100:00100000:3.0:1776189533.887058:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664363840 00000100:00100000:3.0:1776189533.887076:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6910 00000100:00100000:2.0:1776189533.887086:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189533.887116:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b39ea587 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664363584:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189533.887120:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189533.887133:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004ac5ba05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664363136:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189533.887154:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000005ed8ead pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664363520:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189533.887198:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189533.887214:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000023121a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664363776:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189533.887262:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6911 00000100:00100000:2.0:1776189533.887267:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001a690fbd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664363840:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189533.887283:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189533.887321:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001a690fbd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664363840:12345-0@lo:13:osp-pre-1-1.0 Request processed in 53us (321us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189533.887327:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6911 00000100:00100000:2.0:1776189533.887349:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664363648 00000100:00100000:2.0:1776189533.887627:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10070 00000100:00100000:2.0:1776189533.887632:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000403a47d8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664363648:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189533.887670:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000403a47d8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664363648:12345-0@lo:400:kworker/u8:1.0 Request processed in 38us (586us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189533.887678:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10070 00000100:00100000:2.0:1776189533.887714:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002e9e0e35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664363840:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189533.887739:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000023a5b7b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664363648:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189533.887786:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664363072 00000100:00100000:2.0:1776189533.887811:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1513 00000100:00100000:2.0:1776189533.887815:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003e184549 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664363072:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189533.887845:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003e184549 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664363072:12345-0@lo:400:kworker/u8:1.0 Request processed in 29us (812us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189533.887850:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1513 00000100:00100000:1.0:1776189533.888071:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189533.888107:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001a975e34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664363072:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189533.888130:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664363520 00000100:00100000:1.0:1776189533.888151:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664363776 00000100:00100000:3.0:1776189533.888190:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000023fe30ce pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664363200:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776189533.888204:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189533.888236:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000023fe30ce pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664363200:12345-0@lo:13:osp-pre-0-1.0 Request processed in 1156us (3548us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189533.888242:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6910 00000100:00100000:3.0:1776189533.888276:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8642 00000100:00100000:3.0:1776189533.888281:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b7a82fb0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664363520:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189533.888301:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004d2de401 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664363200:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189533.888320:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b7a82fb0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664363520:12345-0@lo:400:kworker/u8:1.0 Request processed in 37us (1121us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189533.888326:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8642 00000100:00100000:3.0:1776189533.888595:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8643 00000100:00100000:3.0:1776189533.888600:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d92ffa01 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664363776:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189533.888639:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189533.888668:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000511337f8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664363712:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189533.888688:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189533.888720:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664363712 00000100:00100000:0.0:1776189533.888743:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000005ed8ead pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664363520:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189533.888750:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8644 00000100:00100000:2.0:1776189533.888756:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000051b336a8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664363712:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189533.888793:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000051b336a8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664363712:12345-0@lo:400:kworker/u8:1.0 Request processed in 37us (104us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189533.888799:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8644 00000100:00100000:2.0:1776189533.888822:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664363328 00000100:00100000:0.0:1776189533.888831:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000511337f8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664363712:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189533.888852:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10071 00000100:00100000:2.0:1776189533.888858:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003a715368 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664363328:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189533.888893:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003a715368 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664363328:12345-0@lo:400:kworker/u8:1.0 Request processed in 35us (256us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189533.888916:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10071 00000100:00100000:3.0:1776189533.889032:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d92ffa01 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664363776:12345-0@lo:400:kworker/u8:1.0 Request processed in 431us (962us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189533.889039:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8643 00000100:00100000:0.0:1776189533.889072:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000048c7c2e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664363328:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189533.889384:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000023121a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664363776:0@lo:400:kworker/u8:1.0 00010000:02020000:0.0:1776189533.917651:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189534.172743:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189534.205568:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189534.428295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189534.684244:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189534.940351:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189535.196347:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189535.229342:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189535.452565:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189535.709377:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189535.965201:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189536.220280:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189536.252455:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189536.476621:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189536.732387:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189536.988318:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189537.244347:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189537.276415:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189537.500302:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189537.661347:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189537.661445:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189537.662098:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654959232 00000100:00100000:3.0:1776189537.662156:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1514 00000100:00100000:3.0:1776189537.662163:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000089ddaa97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654959232:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1776189537.662281:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000089ddaa97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654959232:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 118us (943us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189537.662288:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1514 00000100:00100000:1.0:1776189537.665878:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654959296 00000100:00100000:1.0:1776189537.666025:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10072 00000100:00100000:1.0:1776189537.666032:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004d2de401 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654959296:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776189537.666120:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004d2de401 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654959296:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 87us (4675us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189537.666128:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10072 00000100:00100000:2.0:1776189537.675901:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189537.676168:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654959360 00000100:00100000:1.0:1776189537.676277:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57457 00000100:00100000:1.0:1776189537.676283:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002e9e0e35 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654959360:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:1.0:1776189537.676403:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189537.676513:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002e9e0e35 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654959360:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 228us (618us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189537.676521:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57457 00010000:02020000:0.0:1776189537.756306:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189537.981230:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776189537.981255:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005ee0b9e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664363904:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189537.981328:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189537.981623:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f34aeb18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664363968:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189537.981651:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189537.981800:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664363904 00000100:00100000:2.0:1776189537.981857:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6912 00000100:00100000:2.0:1776189537.981864:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009257ba59 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664363904:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189537.981892:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189537.981950:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009257ba59 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664363904:12345-0@lo:13:osp-pre-0-0.0 Request processed in 85us (622us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189537.981957:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6912 00000100:00100000:2.0:1776189537.981974:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664363968 00000100:00100000:2.0:1776189537.981996:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6913 00000100:00100000:2.0:1776189537.981999:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c6f4c7a3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664363968:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189537.982014:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189537.982044:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c6f4c7a3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664363968:12345-0@lo:13:osp-pre-1-0.0 Request processed in 43us (392us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189537.982050:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6913 00000100:00100000:2.0:1776189537.982091:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f34aeb18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664363968:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189537.982267:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005ee0b9e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664363904:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189538.012288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189538.269427:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189538.300647:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189538.529563:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189538.780334:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189539.004353:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00080000:3.0:1776189539.004356:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189539.004362:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189539.004365:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1776189539.004376:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d3cfd23 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664364032:0@lo:13:osp-pre-0-1.0 00000100:00080000:3.0:1776189539.004376:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9680 00000100:00080000:3.0:1776189539.004381:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189539.004389:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189539.004410:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000048c7c2e3 x1862459664364224/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189539.004451:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1776189539.004453:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1776189539.004458:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189539.004460:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189539.004465:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9680 00000100:00080000:3.0:1776189539.004467:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189539.004472:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189539.004484:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000005ed8ead pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664364288:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189539.004485:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001a975e34 x1862459664364352/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189539.004497:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189539.004500:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189539.004502:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189539.004506:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9680 00000100:00080000:3.0:1776189539.004508:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189539.004511:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189539.004516:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189539.004527:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000023a5b7b3 x1862459664364416/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189539.004542:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f34aeb18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664364160:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189539.004569:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189539.004603:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002fbab827 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664364608:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189539.004611:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000048c7c2e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664364224:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189539.004627:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189539.004646:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:2.0:1776189539.004650:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005ee0b9e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664364096:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189539.004761:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664364288 00000100:00100000:0.0:1776189539.004808:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664364160 00000100:00100000:0.0:1776189539.004822:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664364608 00000100:00100000:0.0:1776189539.004837:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10073 00000100:00100000:0.0:1776189539.004842:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000862ba2d5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664364288:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189539.004917:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189539.004920:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664364032 00000100:00100000:1.0:1776189539.004956:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6914 00000100:00100000:1.0:1776189539.004960:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f9e630d1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664364032:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189539.004974:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189539.005019:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189539.005035:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b39ea587 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664364544:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189539.005087:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189539.005112:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664364096 00000100:00100000:2.0:1776189539.005134:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1515 00000100:00100000:2.0:1776189539.005137:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006e7c9c41 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664364096:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189539.005164:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006e7c9c41 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664364096:12345-0@lo:400:kworker/u8:1.0 Request processed in 26us (145us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189539.005170:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1515 00000100:00100000:2.0:1776189539.005213:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009610fab3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664364672:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189539.005231:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189539.005246:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1776189539.005249:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004ac5ba05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664364480:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189539.005265:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189539.005274:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000068c80116 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664364736:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189539.005287:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189539.005305:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664364544 00000100:00100000:2.0:1776189539.005323:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664364672 00000100:00100000:2.0:1776189539.005333:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664364480 00000100:00100000:2.0:1776189539.005345:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8645 00000100:00100000:2.0:1776189539.005348:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000017cf884a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664364544:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189539.005370:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000017cf884a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664364544:12345-0@lo:400:kworker/u8:1.0 Request processed in 21us (283us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189539.005375:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8645 00000100:00100000:2.0:1776189539.005384:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8646 00000100:00100000:2.0:1776189539.005386:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007ca965f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664364672:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189539.005405:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007ca965f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664364672:12345-0@lo:400:kworker/u8:1.0 Request processed in 18us (174us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189539.005410:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8646 00000100:00100000:2.0:1776189539.005418:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8647 00000100:00100000:2.0:1776189539.005420:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008be8e55e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664364480:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189539.005436:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008be8e55e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664364480:12345-0@lo:400:kworker/u8:1.0 Request processed in 16us (172us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189539.005441:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8647 00000100:00100000:2.0:1776189539.005457:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664364736 00000100:00100000:2.0:1776189539.005467:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6915 00000100:00100000:2.0:1776189539.005471:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b25bee14 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664364736:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189539.005479:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189539.005499:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b25bee14 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664364736:12345-0@lo:13:osp-pre-1-1.0 Request processed in 28us (211us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189539.005504:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6915 00000100:00100000:2.0:1776189539.005541:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005ee0b9e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664364096:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189539.005552:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b39ea587 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664364544:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189539.005572:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004ac5ba05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664364480:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189539.005588:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000068c80116 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664364736:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189539.005594:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009610fab3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664364672:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189539.005652:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664364224 00000100:00100000:2.0:1776189539.005664:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10074 00000100:00100000:2.0:1776189539.005668:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000adb1e111 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664364160:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189539.005695:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000adb1e111 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664364160:12345-0@lo:400:kworker/u8:1.0 Request processed in 26us (1127us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189539.005700:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10074 00000100:00100000:2.0:1776189539.005717:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f34aeb18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664364160:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189539.005729:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10075 00000100:00100000:2.0:1776189539.005732:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004445503c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664364608:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189539.005756:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004445503c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664364608:12345-0@lo:400:kworker/u8:1.0 Request processed in 23us (1131us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189539.005760:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10075 00000100:00100000:2.0:1776189539.005776:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002fbab827 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664364608:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189539.005801:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57458 00000100:00100000:2.0:1776189539.005805:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c944752e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664364224:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189539.005817:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189539.005845:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c944752e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664364224:12345-0@lo:38:kworker/u8:1.0 Request processed in 39us (931us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189539.005850:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57458 00000100:00100000:0.0:1776189539.006902:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000862ba2d5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664364288:12345-0@lo:400:kworker/u8:1.0 Request processed in 2055us (2385us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189539.006913:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10073 00000100:00100000:2.0:1776189539.007081:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000005ed8ead pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664364288:0@lo:400:kworker/u8:1.0 00000100:00080000:3.0:1776189539.008003:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189539.008011:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189539.008017:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000048c7c2e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664364224:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189539.008028:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001a975e34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664364352:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189539.008083:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189539.008113:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f9e630d1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664364032:12345-0@lo:13:osp-pre-0-1.0 Request processed in 3149us (3670us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189539.008114:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000023a5b7b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664364416:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189539.008129:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6914 00000100:00100000:3.0:1776189539.008145:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189539.008201:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664364416 00000100:00100000:1.0:1776189539.008234:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9152 00000100:00100000:1.0:1776189539.008241:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002fbab827 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664364416:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189539.008258:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189539.008318:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664364352 00000100:00100000:0.0:1776189539.008349:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57459 00000100:00100000:0.0:1776189539.008353:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000055462606 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664364352:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189539.008362:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189539.008397:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d3cfd23 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664364032:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189539.008404:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000055462606 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664364352:12345-0@lo:38:kworker/u8:1.0 Request processed in 49us (322us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189539.008412:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57459 00000100:00100000:1.0:1776189539.008515:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002fbab827 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664364416:12345-0@lo:38:kworker/u8:1.0 Request processed in 274us (370us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189539.008524:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9152 00000100:00080000:3.0:1776189539.008539:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189539.008543:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189539.008547:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001a975e34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664364352:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776189539.008556:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189539.008559:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189539.008561:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000023a5b7b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664364416:0@lo:38:kworker/u8:1.0 00010000:02020000:0.0:1776189539.036474:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189539.292279:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189539.324477:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189539.548305:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189539.804325:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189540.060496:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189540.316305:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189540.348662:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189540.573799:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189540.828793:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189541.084279:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189541.340336:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189541.374306:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189541.597607:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189541.852290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189542.108292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189542.364509:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189542.397039:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189542.620286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189542.780426:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189542.780749:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654959424 00000100:00100000:2.0:1776189542.780807:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1516 00000100:00100000:2.0:1776189542.780813:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d3d96460 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654959424:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776189542.781158:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d3d96460 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654959424:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 344us (989us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189542.781166:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1516 00000100:00100000:3.0:1776189542.781622:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189542.781669:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189542.783281:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654959488 00000100:00100000:0.0:1776189542.783354:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654959552 00000100:00100000:0.0:1776189542.783429:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10076 00000100:00100000:0.0:1776189542.783435:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009610fab3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654959488:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1776189542.783507:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009610fab3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654959488:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 71us (2495us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189542.783514:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10076 00000100:00100000:0.0:1776189542.783531:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57460 00000100:00100000:0.0:1776189542.783535:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f326e309 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654959552:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:0.0:1776189542.783549:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189542.783574:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f326e309 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654959552:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 39us (1906us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189542.783580:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57460 00010000:02020000:2.0:1776189542.876367:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189543.100527:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776189543.100552:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000090d63ac9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664364800:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189543.100634:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189543.100679:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000026520930 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664364864:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189543.100712:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189543.100806:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664364800 00000100:00100000:1.0:1776189543.100856:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664364864 00000100:00100000:1.0:1776189543.100879:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6916 00000100:00100000:1.0:1776189543.100884:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e0967c51 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664364800:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189543.100911:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6917 00002000:00000024:1.0:1776189543.100916:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189543.100920:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000071c1b92f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664364864:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189543.100943:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189543.100970:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e0967c51 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664364800:12345-0@lo:13:osp-pre-0-0.0 Request processed in 85us (337us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189543.100979:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6916 00000100:00100000:2.0:1776189543.100986:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000071c1b92f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664364864:12345-0@lo:13:osp-pre-1-0.0 Request processed in 66us (273us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189543.100995:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6917 00000100:00100000:1.0:1776189543.101027:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000026520930 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664364864:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189543.101041:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000090d63ac9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664364800:0@lo:13:osp-pre-0-0.0 00010000:02020000:2.0:1776189543.132300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189543.392242:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189543.424770:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189543.644586:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189543.900305:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189544.125158:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776189544.125197:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000aecac7d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664364928:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189544.125264:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189544.125310:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664364928 00000100:00100000:3.0:1776189544.125367:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6918 00000100:00100000:3.0:1776189544.125374:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001aa39bf8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664364928:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776189544.125398:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00080000:1.0:1776189544.125430:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189544.125436:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189544.125439:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1776189544.125442:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001aa39bf8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664364928:12345-0@lo:13:osp-pre-0-1.0 Request processed in 66us (177us total) trans 0 rc 0/0 00000100:00080000:1.0:1776189544.125446:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9685 00000100:00080000:1.0:1776189544.125451:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1776189544.125454:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6918 00000100:00080000:1.0:1776189544.125458:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189544.125475:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776189544.125479:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000022f2aa27 x1862459664365184/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189544.125487:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000526ed979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664364992:0@lo:13:osp-pre-1-1.0 00000100:00080000:1.0:1776189544.125508:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189544.125512:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189544.125514:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189544.125518:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9685 00000100:00080000:1.0:1776189544.125521:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1776189544.125522:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776189544.125527:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189544.125540:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ab3412dc x1862459664365312/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189544.125545:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000023121a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664365248:0@lo:400:kworker/u8:0.0 00000100:00080000:1.0:1776189544.125557:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189544.125561:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189544.125563:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189544.125566:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9685 00000100:00080000:1.0:1776189544.125568:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189544.125571:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189544.125577:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189544.125586:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e5117e34 x1862459664365376/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189544.125608:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000022f2aa27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664365184:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189544.125639:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189544.125649:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f059006a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664365120:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189544.125656:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ab3412dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664365312:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189544.125691:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189544.125710:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189544.125712:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e5117e34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664365376:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189544.125734:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a829435f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664365568:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189544.125742:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189544.125768:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189544.125783:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664365248 00000100:00100000:1.0:1776189544.125790:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1776189544.125797:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000067c9d69b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664365056:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189544.125811:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664365184 00000100:00100000:3.0:1776189544.125824:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664365312 00000100:00100000:1.0:1776189544.125827:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189544.125833:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664365120 00000100:00100000:1.0:1776189544.125844:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000021503dd8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664365504:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189544.125848:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664365568 00000100:00100000:3.0:1776189544.125865:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10077 00000100:00100000:3.0:1776189544.125869:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba280200 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664365248:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189544.125875:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189544.125906:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba280200 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664365248:12345-0@lo:400:kworker/u8:0.0 Request processed in 36us (330us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189544.125925:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000023121a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664365248:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189544.125937:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776189544.125943:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a93ab5a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664365440:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189544.125965:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189544.126004:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10078 00000100:00100000:1.0:1776189544.126008:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008b54d6c4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664365120:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189544.126040:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008b54d6c4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664365120:12345-0@lo:400:kworker/u8:0.0 Request processed in 31us (332us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189544.126047:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10078 00000100:00100000:1.0:1776189544.126062:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10079 00000100:00100000:1.0:1776189544.126066:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002ef57fef pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664365568:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189544.126090:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002ef57fef pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664365568:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (326us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189544.126096:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10079 00000100:00100000:1.0:1776189544.126108:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57461 00000100:00100000:1.0:1776189544.126111:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002ced68f2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664365184:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189544.126123:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189544.126152:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002ced68f2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664365184:12345-0@lo:38:kworker/u8:0.0 Request processed in 39us (514us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189544.126157:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57461 00000100:00100000:1.0:1776189544.126168:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57462 00000100:00100000:1.0:1776189544.126194:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a78be7d4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664365312:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189544.126201:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189544.126224:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a78be7d4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664365312:12345-0@lo:38:kworker/u8:0.0 Request processed in 52us (533us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189544.126230:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57462 00000100:00100000:1.0:1776189544.126249:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664364992 00000100:00100000:0.0:1776189544.126266:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664365504 00000100:00100000:1.0:1776189544.126276:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6919 00000100:00100000:1.0:1776189544.126282:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a3202503 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664364992:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189544.126295:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189544.126297:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664365440 00000100:00100000:0.0:1776189544.126314:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8648 00000100:00100000:0.0:1776189544.126319:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009b0629a6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664365504:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189544.126331:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a3202503 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664364992:12345-0@lo:13:osp-pre-1-1.0 Request processed in 48us (811us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189544.126338:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6919 00000100:00100000:0.0:1776189544.126360:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009b0629a6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664365504:12345-0@lo:400:kworker/u8:0.0 Request processed in 40us (486us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189544.126366:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8648 00000100:00100000:1.0:1776189544.126377:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000526ed979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664364992:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189544.126382:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8649 00000100:00100000:0.0:1776189544.126387:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009015c790 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664365440:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189544.126410:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000021503dd8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664365504:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189544.126422:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009015c790 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664365440:12345-0@lo:400:kworker/u8:0.0 Request processed in 33us (455us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189544.126426:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f059006a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664365120:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189544.126432:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8649 00000100:00100000:1.0:1776189544.126439:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a829435f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664365568:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189544.126468:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a93ab5a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664365440:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189544.126485:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664365376 00000100:00100000:1.0:1776189544.126509:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9153 00000100:00100000:1.0:1776189544.126515:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000087888c88 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664365376:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189544.126524:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189544.126548:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000087888c88 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664365376:12345-0@lo:38:kworker/u8:0.0 Request processed in 34us (807us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189544.126554:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9153 00000100:00080000:1.0:1776189544.126586:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189544.126589:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189544.126592:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000022f2aa27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664365184:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776189544.126601:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189544.126603:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189544.126606:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ab3412dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664365312:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776189544.126613:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189544.126614:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189544.126616:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e5117e34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664365376:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189544.126811:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000aecac7d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664364928:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189544.126853:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664365056 00000100:00100000:2.0:1776189544.126891:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1517 00000100:00100000:2.0:1776189544.126895:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000027854250 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664365056:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189544.126938:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000027854250 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664365056:12345-0@lo:400:kworker/u8:0.0 Request processed in 42us (1112us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189544.126944:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1517 00000100:00100000:3.0:1776189544.127343:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10077 00000100:00100000:1.0:1776189544.128844:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000067c9d69b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664365056:0@lo:400:kworker/u8:0.0 00010000:02020000:2.0:1776189544.156363:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189544.412279:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189544.444441:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189544.668283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189544.924295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189545.180432:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189545.436292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189545.468405:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189545.692297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189545.948743:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189546.204264:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189546.460292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189546.492744:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189546.716576:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189546.974027:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189547.228286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189547.484302:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189547.516383:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189547.740801:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189547.903099:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189547.903270:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189547.903308:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189547.903346:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654959616 00000100:00100000:2.0:1776189547.903406:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1518 00000100:00100000:2.0:1776189547.903412:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000192d1549 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654959616:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189547.903707:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000192d1549 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654959616:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 293us (437us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189547.903716:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1518 00000100:00100000:1.0:1776189547.913065:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654959744 00000100:00100000:1.0:1776189547.913152:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654959680 00000100:00100000:3.0:1776189547.913405:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10080 00000100:00100000:3.0:1776189547.913413:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008e2382dc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654959680:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189547.913511:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008e2382dc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654959680:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 98us (10203us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189547.913520:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10080 00000100:00100000:3.0:1776189547.913540:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57463 00000100:00100000:3.0:1776189547.913544:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001e34c1c7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654959744:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:3.0:1776189547.913559:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776189547.913587:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001e34c1c7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654959744:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 42us (10497us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189547.913602:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57463 00010000:02020000:0.0:1776189547.996460:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189548.220418:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b385e94b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664365632:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189548.220498:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189548.220696:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003f67f1c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664365696:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189548.220723:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189548.220893:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664365632 00000100:00100000:3.0:1776189548.220938:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664365696 00000100:00100000:3.0:1776189548.220966:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6920 00000100:00100000:3.0:1776189548.220970:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000f61bb34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664365632:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776189548.221003:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189548.221047:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000f61bb34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664365632:12345-0@lo:13:osp-pre-1-0.0 Request processed in 75us (550us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189548.221054:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6920 00000100:00100000:3.0:1776189548.221070:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6921 00000100:00100000:3.0:1776189548.221073:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000aecac7d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664365696:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189548.221089:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189548.221115:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000aecac7d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664365696:12345-0@lo:13:osp-pre-0-0.0 Request processed in 41us (392us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189548.221130:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6921 00000100:00100000:3.0:1776189548.221169:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b385e94b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664365632:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189548.221294:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003f67f1c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664365696:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189548.252295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189548.508282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189548.541190:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189548.767886:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189549.020291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189549.245165:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776189549.245202:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000060a8344d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664365760:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189549.245276:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189549.245674:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664365760 00000100:00100000:2.0:1776189549.245720:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6922 00000100:00100000:2.0:1776189549.245725:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003f67f1c5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664365760:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189549.245762:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189549.245803:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003f67f1c5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664365760:12345-0@lo:13:osp-pre-1-1.0 Request processed in 76us (527us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189549.245810:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6922 00000100:00100000:2.0:1776189549.245863:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776189549.245871:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000053ddeb10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664365824:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189549.245896:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189549.245919:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664365824 00000100:00100000:2.0:1776189549.245939:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6923 00000100:00100000:2.0:1776189549.245942:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b8bc9a84 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664365824:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189549.245951:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189549.245978:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b8bc9a84 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664365824:12345-0@lo:13:osp-pre-0-1.0 Request processed in 34us (82us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189549.245986:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6923 00000100:00100000:2.0:1776189549.246021:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000053ddeb10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664365824:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189549.247105:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000060a8344d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664365760:0@lo:13:osp-pre-1-1.0 00000100:00080000:0.0:1776189549.247335:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189549.247340:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189549.247342:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189549.247347:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9691 00000100:00080000:0.0:1776189549.247360:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776189549.247365:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189549.247382:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000068c80116 x1862459664366016/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189549.247408:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189549.247411:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189549.247412:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189549.247415:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9691 00000100:00080000:0.0:1776189549.247417:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776189549.247419:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189549.247429:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c731e56f x1862459664366144/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189549.247435:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189549.247447:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189549.247449:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189549.247452:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9691 00000100:00080000:0.0:1776189549.247453:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776189549.247456:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189549.247465:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a969b6e4 x1862459664366208/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189549.247505:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000056bf1b01 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664365952:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189549.247550:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189549.247582:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000880386e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664366400:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189549.247601:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189549.247615:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1776189549.247620:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e071bb1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664365888:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189549.247667:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189549.247682:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001b0d11fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664366336:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189549.247702:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189549.248025:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664365888 00000100:00100000:0.0:1776189549.248069:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1519 00000100:00100000:0.0:1776189549.248074:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b64e4d66 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664365888:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189549.248117:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b64e4d66 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664365888:12345-0@lo:400:kworker/u8:1.0 Request processed in 42us (450us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189549.248123:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1519 00000100:00100000:0.0:1776189549.248153:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664366336 00000100:00100000:0.0:1776189549.248204:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8650 00000100:00100000:0.0:1776189549.248207:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009b827fed pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664366336:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189549.248237:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009b827fed pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664366336:12345-0@lo:400:kworker/u8:1.0 Request processed in 29us (536us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189549.248244:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8650 00000100:00100000:0.0:1776189549.248268:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000511337f8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664366080:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189549.248291:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189549.248307:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776189549.248311:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c921b786 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664366272:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189549.248328:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189549.248350:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664365952 00000100:00100000:0.0:1776189549.248372:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664366400 00000100:00100000:0.0:1776189549.248386:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664366080 00000100:00100000:0.0:1776189549.248398:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10081 00000100:00100000:0.0:1776189549.248401:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000098a9e6f8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664365952:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189549.248430:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000098a9e6f8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664365952:12345-0@lo:400:kworker/u8:1.0 Request processed in 28us (881us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189549.248435:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10081 00000100:00100000:0.0:1776189549.248461:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10082 00000100:00100000:0.0:1776189549.248464:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000040e81793 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664366400:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189549.248489:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000040e81793 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664366400:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (888us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189549.248495:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10082 00000100:00100000:0.0:1776189549.248503:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10083 00000100:00100000:0.0:1776189549.248506:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000029eeaafe pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664366080:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189549.248533:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000029eeaafe pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664366080:12345-0@lo:400:kworker/u8:1.0 Request processed in 26us (242us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189549.248538:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10083 00000100:00100000:0.0:1776189549.248557:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664366272 00000100:00100000:0.0:1776189549.248587:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8651 00000100:00100000:0.0:1776189549.248590:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009ea5366f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664366272:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189549.248618:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009ea5366f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664366272:12345-0@lo:400:kworker/u8:1.0 Request processed in 27us (290us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189549.248624:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8651 00000100:00100000:1.0:1776189549.248761:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000068c80116 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664366016:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189549.248789:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189549.248941:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c921b786 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664366272:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189549.248950:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000511337f8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664366080:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189549.248978:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e071bb1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664365888:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189549.248987:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001b0d11fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664366336:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189549.248994:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000056bf1b01 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664365952:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189549.249002:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000880386e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664366400:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189549.253373:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c731e56f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664366144:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189549.253441:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189549.260986:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664366016 00000100:00100000:3.0:1776189549.261038:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664366144 00000100:00100000:3.0:1776189549.261053:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57464 00000100:00100000:3.0:1776189549.261068:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c76be6ad pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664366016:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189549.261086:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189549.261467:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c76be6ad pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664366016:12345-0@lo:38:kworker/u8:1.0 Request processed in 106us (12375us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189549.261480:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57464 00000100:00100000:3.0:1776189549.261496:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57465 00000100:00100000:3.0:1776189549.261499:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cccdb2ad pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664366144:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189549.261506:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189549.261529:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cccdb2ad pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664366144:12345-0@lo:38:kworker/u8:1.0 Request processed in 29us (8090us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189549.261534:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57465 00000100:00080000:1.0:1776189549.263688:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189549.263698:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189549.264374:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c731e56f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664366144:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189549.264390:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a969b6e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664366208:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189549.264454:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776189549.264577:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189549.264582:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189549.264585:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000068c80116 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664366016:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189549.267461:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664366208 00000100:00100000:0.0:1776189549.267520:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9154 00000100:00100000:0.0:1776189549.267527:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c5b17f98 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664366208:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189549.267545:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189549.267636:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c5b17f98 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664366208:12345-0@lo:38:kworker/u8:1.0 Request processed in 109us (3183us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189549.267643:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9154 00000100:00080000:1.0:1776189549.267786:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189549.267791:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189549.267796:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a969b6e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664366208:0@lo:38:kworker/u8:1.0 00010000:02020000:0.0:1776189549.276630:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189549.532306:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189549.564798:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189549.788300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189550.044580:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189550.300292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189550.556325:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189550.588441:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189550.813419:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189551.068415:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189551.324294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189551.581664:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189551.612447:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189551.836298:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189552.092283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189552.348279:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189552.604340:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189552.638011:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189552.860268:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189553.024781:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189553.024821:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189553.024860:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189553.024941:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654959936 00000100:00100000:2.0:1776189553.024985:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654959872 00000100:00100000:2.0:1776189553.025012:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10084 00000100:00100000:2.0:1776189553.025018:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ee307f96 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654959872:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189553.025085:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ee307f96 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654959872:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 67us (264us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189553.025096:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10084 00000100:00100000:2.0:1776189553.025112:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57466 00000100:00100000:2.0:1776189553.025115:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000086d6a2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654959936:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776189553.025129:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189553.025156:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000086d6a2c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654959936:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 39us (383us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189553.025162:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57466 00000100:00100000:1.0:1776189553.025458:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654959808 00000100:00100000:1.0:1776189553.025503:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1520 00000100:00100000:1.0:1776189553.025508:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f37c27d2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654959808:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189553.025536:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f37c27d2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654959808:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 27us (676us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189553.025542:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1520 00010000:02020000:2.0:1776189553.117276:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189553.340509:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776189553.340530:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000090d63ac9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664366464:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189553.340612:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189553.340910:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e310c888 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664366528:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189553.340935:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189553.341083:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664366464 00000100:00100000:3.0:1776189553.341107:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664366528 00000100:00100000:1.0:1776189553.341141:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6924 00000100:00100000:1.0:1776189553.341147:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053ddeb10 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664366464:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189553.341200:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189553.341255:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053ddeb10 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664366464:12345-0@lo:13:osp-pre-0-0.0 Request processed in 107us (643us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189553.341262:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6924 00000100:00100000:1.0:1776189553.341279:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6925 00000100:00100000:1.0:1776189553.341283:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000087513f3f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664366528:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189553.341298:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189553.341320:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000087513f3f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664366528:12345-0@lo:13:osp-pre-1-0.0 Request processed in 36us (385us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189553.341325:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6925 00000100:00100000:0.0:1776189553.341396:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e310c888 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664366528:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189553.341409:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000090d63ac9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664366464:0@lo:13:osp-pre-0-0.0 00010000:02020000:2.0:1776189553.373591:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189553.630679:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189553.660666:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189553.884295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189554.143001:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189554.364465:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005f3b872c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664366592:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189554.364534:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189554.364578:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776189554.364583:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000739e313a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664366656:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189554.364607:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776189554.364693:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189554.364699:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189554.364702:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189554.364708:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9696 00000100:00080000:0.0:1776189554.364712:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00080000:0.0:1776189554.364717:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189554.364736:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ced31f93 x1862459664366848/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189554.364748:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664366592 00000100:00080000:0.0:1776189554.364767:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189554.364771:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189554.364773:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189554.364777:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9696 00000100:00080000:0.0:1776189554.364779:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00080000:0.0:1776189554.364782:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189554.364789:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664366656 00000100:00100000:0.0:1776189554.364793:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e39f5436 x1862459664366976/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189554.364801:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189554.364804:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189554.364806:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189554.364809:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9696 00000100:00100000:2.0:1776189554.364812:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6926 00000100:00080000:0.0:1776189554.364813:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 6s 00000100:00080000:0.0:1776189554.364818:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189554.364822:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000026520930 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664366592:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189554.364832:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d72c99d5 x1862459664367040/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00002000:00000024:2.0:1776189554.364846:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189554.364885:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba6916cf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664366720:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189554.364888:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000026520930 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664366592:12345-0@lo:13:osp-pre-0-1.0 Request processed in 66us (353us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189554.364897:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6926 00000100:00100000:2.0:1776189554.364915:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6927 00000100:00100000:2.0:1776189554.364920:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000090d63ac9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664366656:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189554.364927:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:2.0:1776189554.364931:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189554.364950:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc84a765 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664367168:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189554.364962:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000090d63ac9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664366656:12345-0@lo:13:osp-pre-1-1.0 Request processed in 41us (354us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189554.364969:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6927 00000100:00100000:0.0:1776189554.364980:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189554.365000:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1776189554.365003:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007bd06d4b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664366912:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189554.365008:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000033264082 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664366784:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189554.365040:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189554.365051:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000032656100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664367104:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189554.366358:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189554.366418:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664367168 00000100:00100000:0.0:1776189554.366470:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664367104 00000100:00100000:0.0:1776189554.366490:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8652 00000100:00100000:0.0:1776189554.366494:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000056bf1b01 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664367168:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189554.366534:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000056bf1b01 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664367168:12345-0@lo:400:kworker/u8:0.0 Request processed in 39us (1555us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189554.366540:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8652 00000100:00100000:0.0:1776189554.366556:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8653 00000100:00100000:0.0:1776189554.366559:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e071bb1b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664367104:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189554.366598:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e071bb1b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664367104:12345-0@lo:400:kworker/u8:0.0 Request processed in 38us (245us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189554.366604:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8653 00000100:00100000:0.0:1776189554.366636:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664366720 00000100:00100000:0.0:1776189554.366661:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1521 00000100:00100000:0.0:1776189554.366664:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002e45a4e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664366720:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189554.366696:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002e45a4e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664366720:12345-0@lo:400:kworker/u8:0.0 Request processed in 30us (1770us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189554.366702:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1521 00000100:00100000:0.0:1776189554.366742:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000032656100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664367104:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189554.366766:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189554.366790:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000068c80116 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664367232:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189554.366814:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189554.366832:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba6916cf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664366720:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189554.366841:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc84a765 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664367168:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189554.366864:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664366912 00000100:00100000:0.0:1776189554.366890:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664366784 00000100:00100000:0.0:1776189554.366903:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664367232 00000100:00100000:0.0:1776189554.366916:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10085 00000100:00100000:0.0:1776189554.366919:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000048a53cb2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664366912:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189554.366951:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000048a53cb2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664366912:12345-0@lo:400:kworker/u8:0.0 Request processed in 31us (1912us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189554.366956:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10085 00000100:00100000:0.0:1776189554.366966:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10086 00000100:00100000:0.0:1776189554.366969:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000032656100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664366784:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189554.366995:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000032656100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664366784:12345-0@lo:400:kworker/u8:0.0 Request processed in 25us (229us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189554.367000:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10086 00000100:00100000:0.0:1776189554.367010:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10087 00000100:00100000:0.0:1776189554.367013:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000511337f8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664367232:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189554.367037:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000511337f8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664367232:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (222us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189554.367042:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10087 00000100:00100000:0.0:1776189554.367071:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000033264082 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664366784:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189554.367081:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000068c80116 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664367232:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189554.367122:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ced31f93 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664366848:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189554.367147:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189554.367160:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e39f5436 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664366976:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189554.367812:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000739e313a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664366656:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189554.367836:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f3b872c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664366592:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189554.367865:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007bd06d4b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664366912:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189554.367904:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664366848 00000100:00100000:2.0:1776189554.367932:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57467 00000100:00100000:2.0:1776189554.367936:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000068c80116 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664366848:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189554.367949:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189554.367985:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000068c80116 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664366848:12345-0@lo:38:kworker/u8:0.0 Request processed in 48us (838us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189554.367990:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57467 00000100:00100000:0.0:1776189554.368422:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189554.368676:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d72c99d5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664367040:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189554.368706:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776189554.368798:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189554.368802:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189554.368806:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ced31f93 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664366848:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189554.369129:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664367040 00000100:00100000:1.0:1776189554.369234:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9155 00000100:00100000:1.0:1776189554.369239:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000033264082 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664367040:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189554.369251:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189554.369296:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000033264082 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664367040:12345-0@lo:38:kworker/u8:0.0 Request processed in 56us (590us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189554.369298:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664366976 00000100:00100000:1.0:1776189554.369304:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9155 00000100:00100000:2.0:1776189554.369326:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57468 00000100:00100000:2.0:1776189554.369336:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007bd06d4b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664366976:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189554.369345:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189554.369373:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007bd06d4b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664366976:12345-0@lo:38:kworker/u8:0.0 Request processed in 42us (963us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189554.369380:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57468 00000100:00080000:0.0:1776189554.369410:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189554.369414:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189554.369418:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e39f5436 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664366976:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776189554.369427:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189554.369429:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189554.369432:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d72c99d5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664367040:0@lo:38:kworker/u8:0.0 00010000:02020000:2.0:1776189554.397322:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189554.652288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189554.686220:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189554.908887:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189555.165129:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189555.420288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189555.676357:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189555.711203:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189555.932359:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189556.188465:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189556.444330:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189556.700317:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189556.733143:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189556.956281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189557.212304:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189557.468729:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189557.724460:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189557.756441:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189557.980285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189558.143389:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189558.143437:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189558.143453:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654960064 00000100:00100000:3.0:1776189558.143471:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189558.143540:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654960128 00000100:00100000:3.0:1776189558.143563:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10088 00000100:00100000:3.0:1776189558.143575:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000264516d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654960064:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189558.143649:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000264516d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654960064:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 73us (286us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189558.143666:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10088 00000100:00100000:0.0:1776189558.143872:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654960000 00000100:00100000:0.0:1776189558.143908:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1522 00000100:00100000:0.0:1776189558.143913:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002ea0417b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654960000:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189558.143941:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002ea0417b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654960000:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 27us (505us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189558.143948:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1522 00000100:00100000:2.0:1776189558.144811:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57469 00000100:00100000:2.0:1776189558.144818:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e5e4d83b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654960128:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776189558.144848:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189558.144947:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e5e4d83b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654960128:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 128us (1475us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189558.144954:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57469 00010000:02020000:2.0:1776189558.236315:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189558.461067:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b9092100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664367296:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189558.461134:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189558.461270:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664367296 00000100:00100000:3.0:1776189558.461324:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6928 00000100:00100000:3.0:1776189558.461329:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005bd91e44 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664367296:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776189558.461360:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189558.461419:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005bd91e44 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664367296:12345-0@lo:13:osp-pre-1-0.0 Request processed in 87us (283us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189558.461427:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6928 00000100:00100000:0.0:1776189558.461455:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b696663d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664367360:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189558.461498:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b9092100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664367296:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189558.461589:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189558.461692:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664367360 00000100:00100000:1.0:1776189558.461801:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6929 00000100:00100000:1.0:1776189558.461806:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d72c99d5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664367360:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189558.461824:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189558.461910:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d72c99d5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664367360:12345-0@lo:13:osp-pre-0-0.0 Request processed in 101us (408us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189558.461916:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6929 00000100:00100000:0.0:1776189558.461994:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b696663d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664367360:0@lo:13:osp-pre-0-0.0 00010000:02020000:2.0:1776189558.492294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189558.748643:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189558.783878:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189559.004711:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189559.260314:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:1.0:1776189559.484419:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189559.484426:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189559.484429:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189559.484437:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9701 00000100:00080000:1.0:1776189559.484441:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776189559.484446:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189559.484468:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000739e313a x1862459664367552/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189559.484518:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189559.484522:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189559.484524:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189559.484534:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9701 00000100:00080000:1.0:1776189559.484537:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776189559.484540:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189559.484551:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000060a8344d x1862459664367680/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189559.484560:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189559.484563:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189559.484564:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189559.484567:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9701 00000100:00080000:1.0:1776189559.484568:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776189559.484571:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189559.484581:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b385e94b x1862459664367744/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189559.484639:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a829435f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664367808:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189559.484706:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189559.484733:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000021503dd8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664368064:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189559.484752:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189559.484773:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1776189559.484778:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e310c888 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664367616:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189559.484801:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189559.484816:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a93ab5a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664368000:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189559.484834:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189559.484866:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664367808 00000100:00100000:1.0:1776189559.484902:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664368064 00000100:00100000:1.0:1776189559.484922:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664368000 00000100:00100000:1.0:1776189559.484941:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8654 00000100:00100000:1.0:1776189559.484946:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000067c9d69b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664367808:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189559.484991:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000067c9d69b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664367808:12345-0@lo:400:kworker/u8:0.0 Request processed in 44us (286us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189559.484997:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8654 00000100:00100000:1.0:1776189559.485015:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8655 00000100:00100000:1.0:1776189559.485018:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022f2aa27 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664368064:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189559.485041:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022f2aa27 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664368064:12345-0@lo:400:kworker/u8:0.0 Request processed in 22us (289us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189559.485045:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8655 00000100:00100000:1.0:1776189559.485052:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8656 00000100:00100000:1.0:1776189559.485054:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000526ed979 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664368000:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189559.485067:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000526ed979 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664368000:12345-0@lo:400:kworker/u8:0.0 Request processed in 12us (233us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189559.485072:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8656 00000100:00100000:1.0:1776189559.485096:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664367616 00000100:00100000:1.0:1776189559.485111:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10089 00000100:00100000:1.0:1776189559.485114:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f059006a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664367616:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189559.485130:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f059006a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664367616:12345-0@lo:400:kworker/u8:0.0 Request processed in 16us (331us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189559.485133:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10089 00000100:00100000:1.0:1776189559.485158:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e310c888 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664367616:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189559.485164:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a93ab5a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664368000:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189559.485169:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a829435f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664367808:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189559.485198:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000021503dd8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664368064:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189559.485327:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000822c622a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664367424:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189559.485365:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189559.485392:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e5117e34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664367872:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189559.485412:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189559.485430:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1776189559.485434:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b9092100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664367488:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189559.485455:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189559.485469:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ab3412dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664367936:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189559.485486:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189559.485506:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664367872 00000100:00100000:1.0:1776189559.485609:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664367424 00000100:00100000:1.0:1776189559.485635:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1523 00000100:00100000:1.0:1776189559.485638:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a9501f9f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664367424:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189559.485671:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a9501f9f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664367424:12345-0@lo:400:kworker/u8:0.0 Request processed in 32us (306us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189559.485677:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1523 00000100:00100000:1.0:1776189559.485712:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664367488 00000100:00100000:1.0:1776189559.485730:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664367936 00000100:00100000:1.0:1776189559.485743:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10090 00000100:00100000:1.0:1776189559.485747:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ced31f93 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664367488:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189559.485777:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ced31f93 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664367488:12345-0@lo:400:kworker/u8:0.0 Request processed in 30us (322us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189559.485786:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10090 00000100:00100000:1.0:1776189559.485796:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10091 00000100:00100000:1.0:1776189559.485800:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cc84a765 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664367936:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189559.485822:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cc84a765 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664367936:12345-0@lo:400:kworker/u8:0.0 Request processed in 21us (336us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189559.485826:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10091 00000100:00100000:1.0:1776189559.485867:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b9092100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664367488:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189559.485875:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ab3412dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664367936:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189559.485882:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000822c622a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664367424:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189559.485934:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8657 00000100:00100000:0.0:1776189559.485938:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e39f5436 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664367872:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189559.485980:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e39f5436 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664367872:12345-0@lo:400:kworker/u8:0.0 Request processed in 41us (568us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189559.485986:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8657 00000100:00100000:1.0:1776189559.486052:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e5117e34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664367872:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189559.486153:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000739e313a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664367552:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189559.486200:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189559.486215:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000060a8344d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664367680:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189559.486236:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189559.486250:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b385e94b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664367744:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189559.486272:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189559.486306:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664367552 00000100:00100000:3.0:1776189559.486328:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664367680 00000100:00100000:3.0:1776189559.486342:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57470 00000100:00100000:3.0:1776189559.486356:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000270ed845 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664367552:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189559.486370:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189559.486403:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000270ed845 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664367552:12345-0@lo:38:kworker/u8:0.0 Request processed in 46us (202us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189559.486409:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57470 00000100:00100000:3.0:1776189559.486422:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57471 00000100:00100000:3.0:1776189559.486425:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b696663d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664367680:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189559.486431:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189559.486451:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b696663d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664367680:12345-0@lo:38:kworker/u8:0.0 Request processed in 25us (214us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189559.486456:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57471 00000100:00100000:3.0:1776189559.486474:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664367744 00000100:00100000:3.0:1776189559.486489:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9156 00000100:00100000:3.0:1776189559.486493:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000036a45663 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664367744:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189559.486500:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189559.486522:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000036a45663 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664367744:12345-0@lo:38:kworker/u8:0.0 Request processed in 28us (250us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189559.486714:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005f3b872c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664368192:0@lo:13:osp-pre-1-1.0 00000100:00080000:2.0:1776189559.486737:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189559.486742:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189559.486748:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000739e313a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664367552:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189559.486749:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1776189559.486766:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189559.486769:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189559.486773:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776189559.486773:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000060a8344d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664367680:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189559.486782:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a829435f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664368128:0@lo:13:osp-pre-0-1.0 00000100:00080000:2.0:1776189559.486786:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189559.486789:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189559.486793:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b385e94b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664367744:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189559.486807:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189559.486834:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664368192 00000100:00100000:1.0:1776189559.486849:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664368128 00000100:00100000:2.0:1776189559.486866:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6930 00000100:00100000:2.0:1776189559.486871:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e5117e34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664368192:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189559.486880:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6931 00000100:00100000:1.0:1776189559.486885:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ab3412dc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664368128:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189559.486887:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00002000:00000024:1.0:1776189559.486899:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189559.486935:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e5117e34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664368192:12345-0@lo:13:osp-pre-1-1.0 Request processed in 62us (188us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189559.486942:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ab3412dc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664368128:12345-0@lo:13:osp-pre-0-1.0 Request processed in 55us (136us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189559.486944:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6930 00000100:00100000:1.0:1776189559.486954:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6931 00000100:00100000:2.0:1776189559.486992:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a829435f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664368128:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189559.487002:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f3b872c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664368192:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189559.487355:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9156 00010000:02020000:2.0:1776189559.516303:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189559.772299:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189559.804575:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189560.028309:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189560.284797:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189560.540292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189560.796336:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189560.828416:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189561.053095:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189561.309354:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189561.564294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189561.823413:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189561.852509:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189562.076285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189562.334763:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189562.588989:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189562.844308:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189562.876989:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189563.100298:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189563.260761:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189563.260814:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189563.260849:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189563.260942:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654960256 00000100:00100000:0.0:1776189563.260999:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654960320 00000100:00100000:0.0:1776189563.261015:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10092 00000100:00100000:0.0:1776189563.261021:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005d995c27 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654960256:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1776189563.261091:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005d995c27 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654960256:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 70us (338us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189563.261099:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10092 00000100:00100000:0.0:1776189563.261115:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57472 00000100:00100000:0.0:1776189563.261119:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f0b13a19 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654960320:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:0.0:1776189563.261134:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189563.261160:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f0b13a19 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654960320:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 40us (311us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189563.261166:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57472 00000100:00100000:1.0:1776189563.261535:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654960192 00000100:00100000:1.0:1776189563.261575:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1524 00000100:00100000:1.0:1776189563.261579:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004c8f754f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654960192:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776189563.261621:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004c8f754f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654960192:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 41us (808us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189563.261627:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1524 00010000:02020000:0.0:1776189563.356295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189563.581162:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005f6664e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664368256:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189563.581241:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189563.581287:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776189563.581294:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000021503dd8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664368320:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189563.581315:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189563.581390:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664368256 00000100:00100000:1.0:1776189563.581437:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664368320 00000100:00100000:1.0:1776189563.581467:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6932 00000100:00100000:1.0:1776189563.581473:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba6916cf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664368256:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189563.581479:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6933 00000100:00100000:2.0:1776189563.581486:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a969b6e4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664368320:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189563.581505:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00002000:00000024:2.0:1776189563.581506:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189563.581556:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a969b6e4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664368320:12345-0@lo:13:osp-pre-1-0.0 Request processed in 70us (241us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189563.581563:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6933 00000100:00100000:1.0:1776189563.585504:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba6916cf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664368256:12345-0@lo:13:osp-pre-0-0.0 Request processed in 4021us (4253us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189563.585521:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6932 00000100:00100000:2.0:1776189563.591482:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000021503dd8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664368320:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189563.591519:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f6664e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664368256:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189563.612292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189563.868334:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189563.900442:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189564.124516:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189564.380294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:3.0:1776189564.604617:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189564.604624:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189564.604627:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189564.604636:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9706 00000100:00080000:3.0:1776189564.604640:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1776189564.604645:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189564.604648:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005f6664e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664368384:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189564.604661:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008c04acb9 x1862459664368512/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189564.604713:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1776189564.604721:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1776189564.604740:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776189564.604745:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f40573b9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664368448:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189564.604769:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189564.604800:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664368384 00000100:00100000:1.0:1776189564.604838:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1525 00000100:00100000:1.0:1776189564.604844:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a9f52f65 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664368384:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189564.604885:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a9f52f65 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664368384:12345-0@lo:400:kworker/u8:0.0 Request processed in 41us (173us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189564.604896:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1525 00000100:00100000:1.0:1776189564.604938:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f6664e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664368384:0@lo:400:kworker/u8:0.0 00000100:00080000:3.0:1776189564.604939:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189564.604941:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189564.604944:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9706 00000100:00080000:3.0:1776189564.604946:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1776189564.604949:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189564.604962:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000052e42ae8 x1862459664368640/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189564.604968:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00080000:3.0:1776189564.604971:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189564.604974:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1776189564.604975:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e0c8fd67 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664368576:0@lo:400:kworker/u8:0.0 00000100:00080000:3.0:1776189564.604975:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189564.604978:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9706 00000100:00080000:3.0:1776189564.604980:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1776189564.604982:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189564.604991:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000068da6036 x1862459664368704/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189564.604996:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189564.605012:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3d6c201 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664368768:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189564.605032:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189564.605049:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038a7e045 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664369024:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189564.605053:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664368448 00000100:00100000:1.0:1776189564.605069:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189564.605080:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776189564.605080:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664368576 00000100:00100000:1.0:1776189564.605084:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003cd25a35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664368960:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189564.605097:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10093 00000100:00100000:3.0:1776189564.605101:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a93ab5a8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664368448:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189564.605104:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189564.605130:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664368768 00000100:00100000:3.0:1776189564.605131:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a93ab5a8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664368448:12345-0@lo:400:kworker/u8:0.0 Request processed in 29us (361us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189564.605137:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10093 00000100:00100000:3.0:1776189564.605148:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10094 00000100:00100000:1.0:1776189564.605149:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664369024 00000100:00100000:3.0:1776189564.605152:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000021503dd8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664368576:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189564.605163:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664368960 00000100:00100000:3.0:1776189564.605205:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000021503dd8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664368576:12345-0@lo:400:kworker/u8:0.0 Request processed in 52us (209us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189564.605211:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10094 00000100:00100000:2.0:1776189564.605462:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a45b748c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664368896:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189564.605494:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189564.605515:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c04acb9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664368512:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189564.605515:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776189564.605523:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006302d616 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664368832:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189564.605557:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189564.605561:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189564.605604:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8658 00000100:00100000:1.0:1776189564.605610:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f6664e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664368768:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189564.605748:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052e42ae8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664368640:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189564.607098:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f6664e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664368768:12345-0@lo:400:kworker/u8:0.0 Request processed in 1483us (2060us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189564.607109:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8658 00000100:00100000:0.0:1776189564.607211:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189564.607257:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000068da6036 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664368704:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189564.607284:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189564.612301:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776189564.612320:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002c2b2af4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664369088:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189564.612379:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189564.612410:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664369088 00000100:00100000:2.0:1776189564.612473:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6934 00000100:00100000:2.0:1776189564.612478:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e8ddaf27 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3364:x1862459664369088:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189564.612501:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189564.612548:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e8ddaf27 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3364:x1862459664369088:12345-0@lo:13:osp-pre-1-1.0 Request processed in 69us (171us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189564.612555:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6934 00000100:00100000:2.0:1776189564.612597:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002c2b2af4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664369088:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189564.612737:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664368896 00000100:00100000:2.0:1776189564.612764:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664368512 00000100:00100000:2.0:1776189564.612781:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664368640 00000100:00100000:2.0:1776189564.612790:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10095 00000100:00100000:2.0:1776189564.612794:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f3b872c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664368896:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189564.612842:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f3b872c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664368896:12345-0@lo:400:kworker/u8:0.0 Request processed in 46us (7348us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189564.612848:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10095 00000100:00100000:2.0:1776189564.612859:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57473 00000100:00100000:2.0:1776189564.612862:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c731e56f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664368512:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189564.612875:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189564.612905:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c731e56f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664368512:12345-0@lo:38:kworker/u8:0.0 Request processed in 41us (7349us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189564.612909:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57473 00000100:00100000:2.0:1776189564.612918:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57474 00000100:00100000:2.0:1776189564.612921:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000880386e2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664368640:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189564.612928:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189564.612948:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000880386e2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664368640:12345-0@lo:38:kworker/u8:0.0 Request processed in 26us (5738us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189564.612953:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57474 00000100:00100000:0.0:1776189564.614958:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664368832 00000100:00100000:1.0:1776189564.614981:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8659 00000100:00100000:1.0:1776189564.614991:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c0d5b1aa pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664369024:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189564.615017:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8660 00000100:00100000:0.0:1776189564.615023:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b385e94b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664368960:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189564.615070:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c0d5b1aa pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664369024:12345-0@lo:400:kworker/u8:0.0 Request processed in 80us (10000us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189564.615080:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b385e94b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664368960:12345-0@lo:400:kworker/u8:0.0 Request processed in 55us (9973us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189564.615086:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8659 00000100:00100000:0.0:1776189564.615093:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8660 00000100:00100000:0.0:1776189564.615124:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8661 00000100:00100000:0.0:1776189564.615129:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005c6b517b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664368832:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189564.615160:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005c6b517b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664368832:12345-0@lo:400:kworker/u8:0.0 Request processed in 31us (9602us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189564.615166:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8661 00000100:00100000:0.0:1776189564.615211:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664368704 00000100:00100000:0.0:1776189564.615243:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9157 00000100:00100000:0.0:1776189564.615248:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001b0d11fc pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664368704:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189564.615261:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189564.615293:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001b0d11fc pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664368704:12345-0@lo:38:kworker/u8:0.0 Request processed in 45us (8009us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189564.615300:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9157 00000100:00080000:0.0:1776189564.615350:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189564.615353:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189564.615358:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c04acb9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664368512:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776189564.615373:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189564.615375:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189564.615379:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052e42ae8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664368640:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776189564.615387:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189564.615390:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189564.615392:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000068da6036 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664368704:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189564.615476:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f40573b9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664368448:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189564.615495:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776189564.615503:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b0629a6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664369152:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189564.615513:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003cd25a35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664368960:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189564.615527:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038a7e045 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664369024:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189564.615537:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189564.615539:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3d6c201 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664368768:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189564.615624:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e0c8fd67 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664368576:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189564.615652:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664369152 00000100:00100000:0.0:1776189564.615673:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6935 00000100:00100000:0.0:1776189564.615676:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009015c790 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664369152:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776189564.615687:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189564.618383:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006302d616 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664368832:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189564.618404:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a45b748c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664368896:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189564.618422:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009015c790 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664369152:12345-0@lo:13:osp-pre-0-1.0 Request processed in 2742us (2884us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189564.618435:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6935 00000100:00100000:1.0:1776189564.618607:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b0629a6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664369152:0@lo:13:osp-pre-0-1.0 00010000:02020000:0.0:1776189564.636320:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189564.892353:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189564.924401:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189565.148404:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189565.404301:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189565.660259:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189565.917605:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189565.948489:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189566.172338:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189566.428304:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189566.685707:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189566.940622:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189566.972921:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189567.196822:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189567.453816:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189567.708320:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189567.965623:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189567.997482:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189568.220239:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189568.379596:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189568.379661:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189568.379708:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189568.379822:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654960512 00000100:00100000:1.0:1776189568.379831:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654960384 00000100:00100000:2.0:1776189568.379865:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654960448 00000100:00100000:1.0:1776189568.379873:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1526 00000100:00100000:1.0:1776189568.379881:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000be36d03e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654960384:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189568.379893:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10096 00000100:00100000:2.0:1776189568.379899:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006302d616 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654960448:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189568.379959:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000be36d03e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654960384:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 77us (250us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189568.379967:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1526 00000100:00100000:1.0:1776189568.380015:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57475 00000100:00100000:1.0:1776189568.380020:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a45b748c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654960512:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776189568.380046:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189568.383966:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006302d616 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654960448:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 4056us (4295us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189568.383983:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10096 00000100:00100000:1.0:1776189568.385823:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a45b748c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654960512:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 5795us (6230us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189568.385838:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57475 00010000:02020000:2.0:1776189568.478934:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189568.700699:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e0c8fd67 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664369280:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189568.700766:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189568.700813:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776189568.700819:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b0629a6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664369216:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189568.700841:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189568.700962:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664369280 00000100:00100000:3.0:1776189568.701014:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664369216 00000100:00100000:3.0:1776189568.701038:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6936 00000100:00100000:3.0:1776189568.701043:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c921b786 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664369280:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189568.701074:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189568.701116:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c921b786 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664369280:12345-0@lo:13:osp-pre-0-0.0 Request processed in 72us (350us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189568.701124:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6936 00000100:00100000:0.0:1776189568.701254:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6937 00000100:00100000:0.0:1776189568.701258:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000375d4d18 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664369216:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189568.701275:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189568.701316:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000375d4d18 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664369216:12345-0@lo:13:osp-pre-1-0.0 Request processed in 56us (474us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189568.701321:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6937 00000100:00100000:0.0:1776189568.701390:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b0629a6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664369216:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189568.701404:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e0c8fd67 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664369280:0@lo:13:osp-pre-0-0.0 00010000:02020000:2.0:1776189568.735980:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189568.992572:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189569.023869:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189569.244297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189569.500286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:3.0:1776189569.724383:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189569.724390:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189569.724393:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189569.724401:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9711 00000100:00080000:3.0:1776189569.724406:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189569.724411:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189569.724429:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003cd25a35 x1862459664369472/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189569.724458:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189569.724461:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189569.724462:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189569.724465:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9711 00000100:00080000:3.0:1776189569.724467:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189569.724470:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189569.724479:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000068da6036 x1862459664369600/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189569.724486:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189569.724489:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189569.724490:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189569.724492:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9711 00000100:00080000:3.0:1776189569.724494:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189569.724496:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189569.724507:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000052e42ae8 x1862459664369664/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189569.724565:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f40573b9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664369536:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189569.724631:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189569.724656:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e532ce7c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664369920:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189569.724682:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189569.724704:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1776189569.724708:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c04acb9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664369728:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189569.724726:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189569.724735:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c68cc49e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664369984:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189569.724749:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189569.724778:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664369920 00000100:00100000:3.0:1776189569.724816:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664369728 00000100:00100000:3.0:1776189569.724831:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664369984 00000100:00100000:3.0:1776189569.724848:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8662 00000100:00100000:3.0:1776189569.724853:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005cdf5f34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664369920:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189569.724897:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005cdf5f34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664369920:12345-0@lo:400:kworker/u8:1.0 Request processed in 43us (215us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189569.724904:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8662 00000100:00100000:3.0:1776189569.724920:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8663 00000100:00100000:3.0:1776189569.724923:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d5cfaed8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664369728:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189569.724947:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d5cfaed8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664369728:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (221us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189569.724952:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8663 00000100:00100000:3.0:1776189569.724963:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8664 00000100:00100000:3.0:1776189569.724966:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007f770c5d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664369984:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189569.724988:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007f770c5d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664369984:12345-0@lo:400:kworker/u8:1.0 Request processed in 21us (239us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189569.724993:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8664 00000100:00100000:3.0:1776189569.725022:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c04acb9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664369728:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189569.725032:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c68cc49e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664369984:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189569.725039:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e532ce7c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664369920:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189569.725136:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038a7e045 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664369408:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189569.725166:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189569.725216:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000838deece pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664369856:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189569.725235:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189569.725253:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1776189569.725258:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3d6c201 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664369344:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189569.725282:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189569.725297:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f8dcf939 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664369792:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189569.725316:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189569.725338:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003cd25a35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664369472:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189569.725359:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189569.725374:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000068da6036 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664369600:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189569.725392:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189569.725404:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052e42ae8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664369664:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189569.725425:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189569.725451:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664369344 00000100:00100000:0.0:1776189569.725474:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1527 00000100:00100000:0.0:1776189569.725477:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000816fca97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664369344:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189569.725507:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000816fca97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664369344:12345-0@lo:400:kworker/u8:1.0 Request processed in 28us (224us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189569.725513:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1527 00000100:00100000:0.0:1776189569.725540:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664369536 00000100:00100000:0.0:1776189569.725558:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664369408 00000100:00100000:0.0:1776189569.725567:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664369856 00000100:00100000:0.0:1776189569.725575:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664369472 00000100:00100000:0.0:1776189569.725582:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664369600 00000100:00100000:0.0:1776189569.725591:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10097 00000100:00100000:0.0:1776189569.725595:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e0c8fd67 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664369536:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189569.725626:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e0c8fd67 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664369536:12345-0@lo:400:kworker/u8:1.0 Request processed in 31us (996us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189569.725632:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10097 00000100:00100000:0.0:1776189569.725642:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10098 00000100:00100000:0.0:1776189569.725645:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007c850b41 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664369408:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189569.725669:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007c850b41 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664369408:12345-0@lo:400:kworker/u8:1.0 Request processed in 23us (502us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189569.725674:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10098 00000100:00100000:0.0:1776189569.725684:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10099 00000100:00100000:0.0:1776189569.725687:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a8591738 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664369856:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189569.725710:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a8591738 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664369856:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (475us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189569.725715:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10099 00000100:00100000:0.0:1776189569.725725:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57476 00000100:00100000:0.0:1776189569.725729:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000089232900 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664369472:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189569.725741:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189569.725769:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000089232900 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664369472:12345-0@lo:38:kworker/u8:1.0 Request processed in 39us (409us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189569.725774:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57476 00000100:00100000:0.0:1776189569.725784:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57477 00000100:00100000:0.0:1776189569.725787:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000031b5a624 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664369600:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189569.725792:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189569.725810:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000031b5a624 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664369600:12345-0@lo:38:kworker/u8:1.0 Request processed in 23us (418us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189569.725816:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57477 00000100:00100000:0.0:1776189569.725835:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664369664 00000100:00100000:0.0:1776189569.725852:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9158 00000100:00100000:0.0:1776189569.725855:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000748b663e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664369664:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189569.725862:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189569.725884:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000748b663e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664369664:12345-0@lo:38:kworker/u8:1.0 Request processed in 28us (459us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189569.725889:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9158 00000100:00080000:0.0:1776189569.725918:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189569.725921:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189569.725924:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003cd25a35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664369472:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776189569.725932:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189569.725934:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189569.725937:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000068da6036 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664369600:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776189569.725943:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189569.725945:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189569.725947:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052e42ae8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664369664:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189569.725976:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3d6c201 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664369344:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189569.725986:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038a7e045 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664369408:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189569.725993:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000838deece pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664369856:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189569.726014:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664369792 00000100:00100000:0.0:1776189569.726032:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8665 00000100:00100000:0.0:1776189569.726035:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cd150e5b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664369792:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189569.726061:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cd150e5b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664369792:12345-0@lo:400:kworker/u8:1.0 Request processed in 25us (745us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189569.726066:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8665 00000100:00100000:0.0:1776189569.726082:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f8dcf939 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664369792:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189569.726381:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f40573b9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664369536:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189569.726774:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776189569.726849:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776189569.726861:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a543fbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664370112:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189569.726901:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189569.726927:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664370112 00000100:00100000:2.0:1776189569.726954:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6938 00000100:00100000:2.0:1776189569.726958:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002c2b2af4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664370112:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189569.726974:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189569.727020:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002c2b2af4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664370112:12345-0@lo:13:osp-pre-0-1.0 Request processed in 60us (119us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189569.727026:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6938 00000100:00100000:2.0:1776189569.727054:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a543fbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664370112:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189569.729152:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005c6d79ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664370048:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189569.729227:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189569.730165:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664370048 00000100:00100000:2.0:1776189569.730270:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6939 00000100:00100000:2.0:1776189569.730275:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000060a8344d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664370048:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189569.730301:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189569.730371:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000060a8344d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664370048:12345-0@lo:13:osp-pre-1-1.0 Request processed in 95us (1145us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189569.730379:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6939 00000100:00100000:0.0:1776189569.730454:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005c6d79ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664370048:0@lo:13:osp-pre-1-1.0 00010000:02020000:0.0:1776189569.757062:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189570.013203:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189570.044590:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189570.268314:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189570.524363:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189570.780338:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189571.036276:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189571.069469:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189571.292841:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189571.548805:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189571.804295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189572.060379:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189572.095093:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189572.316298:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189572.572303:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189572.828289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189573.084288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189573.116540:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189573.340688:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189573.498934:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189573.499220:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189573.499297:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189573.499363:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654960640 00000100:00100000:1.0:1776189573.499507:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654960576 00000100:00100000:1.0:1776189573.499553:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1528 00000100:00100000:1.0:1776189573.499559:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c4f71e57 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654960576:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1776189573.499585:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654960704 00000100:00100000:1.0:1776189573.499638:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c4f71e57 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654960576:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 77us (711us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189573.499646:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1528 00000100:00100000:2.0:1776189573.502882:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10100 00000100:00100000:2.0:1776189573.502895:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005be3c9f3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654960640:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776189573.503444:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005be3c9f3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654960640:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 549us (4224us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189573.503461:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10100 00000100:00100000:2.0:1776189573.503491:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57478 00000100:00100000:2.0:1776189573.503495:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cc4b762b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654960704:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:2.0:1776189573.503513:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189573.503543:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cc4b762b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654960704:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 47us (4246us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189573.503549:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57478 00010000:02020000:2.0:1776189573.596408:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189573.820500:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776189573.820525:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d52de5d9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664370176:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189573.820594:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189573.820724:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664370176 00000100:00100000:0.0:1776189573.820816:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000739e313a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664370240:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189573.820816:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6940 00000100:00100000:3.0:1776189573.820825:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007c812403 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664370176:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189573.820850:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:3.0:1776189573.820859:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189573.820902:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007c812403 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664370176:12345-0@lo:13:osp-pre-0-0.0 Request processed in 77us (308us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189573.820921:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6940 00000100:00100000:3.0:1776189573.821247:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664370240 00000100:00100000:0.0:1776189573.821266:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d52de5d9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664370176:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189573.821287:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6941 00000100:00100000:2.0:1776189573.821293:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000412a95e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664370240:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189573.821312:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189573.821355:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000412a95e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664370240:12345-0@lo:13:osp-pre-1-0.0 Request processed in 61us (506us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189573.821362:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6941 00000100:00100000:0.0:1776189573.821387:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000739e313a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664370240:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189573.852551:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189574.108781:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189574.141982:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189574.364683:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189574.620293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189574.845028:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776189574.845059:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d9eb44b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664370304:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189574.845155:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189574.845274:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664370304 00000100:00100000:2.0:1776189574.845342:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6942 00000100:00100000:2.0:1776189574.845351:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000822c622a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664370304:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189574.845384:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189574.845444:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000822c622a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664370304:12345-0@lo:13:osp-pre-0-1.0 Request processed in 93us (291us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189574.845452:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6942 00000100:00100000:0.0:1776189574.845548:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d9eb44b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664370304:0@lo:13:osp-pre-0-1.0 00000100:00080000:0.0:1776189574.845997:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189574.846002:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189574.846006:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189574.846021:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9716 00000100:00080000:0.0:1776189574.846024:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1776189574.846028:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189574.846048:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e9c407a2 x1862459664370560/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189574.846054:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d9eb44b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664370432:0@lo:400:kworker/u8:1.0 00000100:00080000:0.0:1776189574.846071:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189574.846074:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189574.846075:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189574.846078:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9716 00000100:00080000:0.0:1776189574.846080:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1776189574.846083:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189574.846092:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e0bab4ab x1862459664370688/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189574.846099:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189574.846102:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189574.846103:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1776189574.846104:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776189574.846107:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9716 00000100:00080000:0.0:1776189574.846108:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1776189574.846113:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189574.846121:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f09d90e0 x1862459664370752/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189574.846124:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776189574.846128:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e42142fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664370496:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189574.846151:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189574.846164:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a543fbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664370880:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189574.846189:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000201d0f3c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664370944:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189574.846206:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189574.846214:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189574.846236:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664370880 00000100:00100000:0.0:1776189574.846238:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9c407a2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664370560:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189574.846260:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189574.846267:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8666 00000100:00100000:0.0:1776189574.846271:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e0bab4ab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664370688:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189574.846271:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e310c888 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664370880:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189574.846290:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189574.846304:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f09d90e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664370752:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189574.846306:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e310c888 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664370880:12345-0@lo:400:kworker/u8:1.0 Request processed in 34us (100us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189574.846312:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8666 00000100:00100000:0.0:1776189574.846325:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189574.846340:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664370432 00000100:00100000:0.0:1776189574.846353:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664370496 00000100:00100000:0.0:1776189574.846378:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664370944 00000100:00100000:1.0:1776189574.846380:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1529 00000100:00100000:1.0:1776189574.846384:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b1a6ecdc pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664370432:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189574.846393:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664370560 00000100:00100000:0.0:1776189574.846403:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664370688 00000100:00100000:1.0:1776189574.846413:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b1a6ecdc pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664370432:12345-0@lo:400:kworker/u8:1.0 Request processed in 27us (309us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189574.846415:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10101 00000100:00100000:0.0:1776189574.846419:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b9092100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664370496:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189574.846419:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1529 00000100:00100000:0.0:1776189574.846450:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b9092100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664370496:12345-0@lo:400:kworker/u8:1.0 Request processed in 30us (299us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189574.846456:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10101 00000100:00100000:1.0:1776189574.846456:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a543fbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664370880:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189574.846464:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e42142fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664370496:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189574.846467:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10102 00000100:00100000:0.0:1776189574.846471:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005ab67562 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664370944:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189574.846471:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d9eb44b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664370432:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189574.846498:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005ab67562 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664370944:12345-0@lo:400:kworker/u8:1.0 Request processed in 26us (284us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189574.846501:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000058863251 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664370368:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189574.846504:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10102 00000100:00100000:0.0:1776189574.846514:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57479 00000100:00100000:0.0:1776189574.846517:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000051b336a8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664370560:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189574.846521:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:02000000:0.0:1776189574.846533:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189574.846533:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000733a9e44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664370816:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189574.846549:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189574.846559:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000051b336a8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664370560:12345-0@lo:38:kworker/u8:1.0 Request processed in 41us (300us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189574.846561:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1776189574.846565:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57479 00000100:00100000:1.0:1776189574.846566:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001706ae06 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664370624:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189574.846574:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57480 00000100:00100000:0.0:1776189574.846577:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000056bf1b01 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664370688:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189574.846582:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:02000000:0.0:1776189574.846584:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189574.846589:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009fddac3a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664371008:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189574.846602:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000056bf1b01 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664370688:12345-0@lo:38:kworker/u8:1.0 Request processed in 24us (312us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189574.846604:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189574.846608:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57480 00000100:00100000:0.0:1776189574.846617:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664370624 00000100:00100000:1.0:1776189574.846621:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664370816 00000100:00100000:0.0:1776189574.846629:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10103 00000100:00100000:0.0:1776189574.846633:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d9eb44b4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664370624:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189574.846638:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664371008 00000100:00100000:1.0:1776189574.846655:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8667 00000100:00100000:1.0:1776189574.846658:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009b0629a6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664370816:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189574.846735:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664370368 00000100:00100000:2.0:1776189574.846760:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6943 00000100:00100000:2.0:1776189574.846764:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000739e313a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3364:x1862459664370368:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189574.846778:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189574.846889:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664370752 00000100:00100000:3.0:1776189574.846912:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9159 00000100:00100000:3.0:1776189574.846915:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e071bb1b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664370752:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189574.846923:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189574.847486:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009b0629a6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664370816:12345-0@lo:400:kworker/u8:1.0 Request processed in 825us (934us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189574.847498:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8667 00000100:00100000:1.0:1776189574.847517:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8668 00000100:00100000:1.0:1776189574.847521:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e42142fc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+6:3364:x1862459664371008:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189574.847557:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e42142fc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+6:3364:x1862459664371008:12345-0@lo:400:kworker/u8:1.0 Request processed in 35us (952us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189574.847563:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8668 00000100:00080000:1.0:1776189574.847629:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189574.847633:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189574.847637:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9c407a2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664370560:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776189574.847647:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189574.847650:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189574.847652:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e0bab4ab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664370688:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189574.847684:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001706ae06 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664370624:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189574.847694:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009fddac3a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664371008:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189574.847704:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000733a9e44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664370816:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189574.847755:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000201d0f3c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664370944:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189574.850198:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000739e313a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664370368:12345-0@lo:13:osp-pre-1-1.0 Request processed in 3426us (3669us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189574.850214:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6943 00000100:00100000:3.0:1776189574.850349:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e071bb1b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664370752:12345-0@lo:38:kworker/u8:1.0 Request processed in 3431us (4022us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189574.850362:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9159 00000100:00080000:2.0:1776189574.850456:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189574.850461:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189574.850466:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f09d90e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664370752:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189574.850577:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000058863251 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664370368:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189574.853232:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d9eb44b4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664370624:12345-0@lo:400:kworker/u8:1.0 Request processed in 6590us (6641us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189574.853248:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10103 00010000:02020000:0.0:1776189574.877140:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189575.132450:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189575.164640:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189575.388413:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189575.644310:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189575.900283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189576.156298:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189576.188419:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189576.412686:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189576.668794:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189576.924368:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189577.182482:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189577.215448:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189577.436313:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189577.692307:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189577.948381:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189578.204307:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189578.236353:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189578.460283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189578.620264:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189578.620320:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189578.620416:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654960832 00000100:00100000:0.0:1776189578.620474:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10104 00000100:00100000:0.0:1776189578.620480:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cc8ba9ef pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654960832:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1776189578.620549:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cc8ba9ef pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654960832:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 68us (292us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189578.620567:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10104 00000100:00100000:2.0:1776189578.620705:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654960768 00000100:00100000:2.0:1776189578.620742:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1530 00000100:00100000:2.0:1776189578.620747:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000090683405 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654960768:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776189578.620774:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000090683405 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654960768:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 27us (455us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189578.620781:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1530 00000100:00100000:1.0:1776189578.632518:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189578.632695:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654960896 00000100:00100000:3.0:1776189578.632986:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57481 00000100:00100000:3.0:1776189578.632993:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000733a9e44 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654960896:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:3.0:1776189578.633012:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776189578.633085:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000733a9e44 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654960896:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 92us (575us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189578.633093:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57481 00010000:02020000:0.0:1776189578.716327:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189578.940722:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776189578.940745:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee179fa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664371072:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189578.940894:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189578.941018:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776189578.941027:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001706ae06 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664371136:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189578.941055:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189578.944014:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664371072 00000100:00100000:3.0:1776189578.944460:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664371136 00000100:00100000:1.0:1776189578.944505:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6944 00000100:00100000:1.0:1776189578.944512:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009fddac3a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664371072:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189578.944546:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189578.944619:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009fddac3a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664371072:12345-0@lo:13:osp-pre-0-0.0 Request processed in 106us (3726us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189578.944626:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6944 00000100:00100000:1.0:1776189578.944651:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6945 00000100:00100000:1.0:1776189578.944655:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e0bab4ab pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664371136:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189578.944669:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189578.944701:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e0bab4ab pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664371136:12345-0@lo:13:osp-pre-1-0.0 Request processed in 45us (3646us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189578.944706:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6945 00000100:00100000:1.0:1776189578.944756:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001706ae06 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664371136:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189578.944841:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee179fa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664371072:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189578.972270:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189579.228327:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189579.260403:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189579.484395:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189579.741731:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:0.0:1776189579.964357:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189579.964364:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189579.964367:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189579.964375:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9721 00000100:00080000:0.0:1776189579.964379:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776189579.964384:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189579.964405:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d02fb24b x1862459664371328/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189579.964423:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189579.964426:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189579.964427:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189579.964430:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9721 00000100:00080000:0.0:1776189579.964432:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776189579.964435:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189579.964443:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000053ba7e65 x1862459664371520/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189579.964450:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1776189579.964450:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bf60fbf3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664371264:0@lo:400:kworker/u8:0.0 00000100:00080000:0.0:1776189579.964453:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189579.964454:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189579.964456:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9721 00000100:00080000:0.0:1776189579.964458:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776189579.964460:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189579.964468:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b7a82fb0 x1862459664371584/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189579.964506:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189579.964551:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000067c9d69b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664371712:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189579.964574:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189579.964588:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1776189579.964592:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001706ae06 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664371392:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189579.964613:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189579.964625:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000022f2aa27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664371776:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189579.964642:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189579.964668:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664371264 00000100:00100000:1.0:1776189579.964668:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664371392 00000100:00100000:0.0:1776189579.964698:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664371776 00000100:00100000:1.0:1776189579.964702:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6946 00000100:00100000:1.0:1776189579.964708:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f09d90e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664371392:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189579.964711:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10105 00000100:00100000:0.0:1776189579.964714:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e9c407a2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664371264:12345-0@lo:400:kworker/u8:0.0 00002000:00000024:1.0:1776189579.964726:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189579.964799:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009820cd1e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664371456:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189579.964839:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e9c407a2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664371264:12345-0@lo:400:kworker/u8:0.0 Request processed in 123us (333us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189579.964846:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10105 00000100:00100000:0.0:1776189579.964863:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10106 00000100:00100000:0.0:1776189579.964866:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000201d0f3c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664371776:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189579.967011:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189579.967048:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000526ed979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664371840:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189579.967081:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189579.967184:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:2.0:1776189579.967260:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017cf884a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664371200:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189579.967287:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189579.967313:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d92ffa01 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664371648:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189579.967331:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189579.967342:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b827fed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664371904:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189579.967690:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189579.967777:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776189579.967785:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a7b82f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664371968:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189579.967810:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189579.967830:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02fb24b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664371328:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189579.967849:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189579.967860:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000053ba7e65 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664371520:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189579.967875:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189579.967881:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b7a82fb0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664371584:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189579.967898:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189579.967944:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664371712 00000100:00100000:2.0:1776189579.967979:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664371840 00000100:00100000:2.0:1776189579.967992:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664371648 00000100:00100000:2.0:1776189579.968003:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664371904 00000100:00100000:2.0:1776189579.968018:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8669 00000100:00100000:2.0:1776189579.968023:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007a543fbe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664371712:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189579.970355:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f09d90e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3363:x1862459664371392:12345-0@lo:13:osp-pre-0-1.0 Request processed in 5638us (5733us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189579.970370:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6946 00000100:00100000:1.0:1776189579.970405:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664371968 00000100:00100000:1.0:1776189579.970439:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6947 00000100:00100000:1.0:1776189579.970444:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000540d168e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3364:x1862459664371968:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189579.970465:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189579.970513:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000540d168e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3364:x1862459664371968:12345-0@lo:13:osp-pre-1-1.0 Request processed in 68us (2702us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189579.970519:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6947 00000100:00100000:1.0:1776189579.970880:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001706ae06 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664371392:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189579.975884:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000201d0f3c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664371776:12345-0@lo:400:kworker/u8:0.0 Request processed in 11008us (11233us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189579.975915:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10106 00000100:00100000:0.0:1776189579.976101:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664371456 00000100:00100000:0.0:1776189579.976132:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664371328 00000100:00100000:0.0:1776189579.976144:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664371520 00000100:00100000:0.0:1776189579.976156:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10107 00000100:00100000:0.0:1776189579.976162:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000006c1656a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664371456:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189579.976190:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57482 00000100:00100000:3.0:1776189579.976197:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a109d002 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664371328:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189579.976213:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189579.976272:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a109d002 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664371328:12345-0@lo:38:kworker/u8:0.0 Request processed in 73us (8421us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189579.976277:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000006c1656a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664371456:12345-0@lo:400:kworker/u8:0.0 Request processed in 111us (9269us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189579.976284:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57482 00000100:00100000:2.0:1776189579.976667:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007a543fbe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664371712:12345-0@lo:400:kworker/u8:0.0 Request processed in 8641us (12090us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189579.976676:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8669 00000100:00100000:2.0:1776189579.976736:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009820cd1e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664371456:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189579.976770:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664371200 00000100:00100000:2.0:1776189579.976796:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1531 00000100:00100000:2.0:1776189579.976800:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b90a26ba pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664371200:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189579.976836:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b90a26ba pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664371200:12345-0@lo:400:kworker/u8:0.0 Request processed in 36us (9549us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189579.976843:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1531 00000100:00100000:0.0:1776189579.977819:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10107 00000100:00100000:0.0:1776189579.977850:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57483 00000100:00100000:0.0:1776189579.977855:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000decf8864 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664371520:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189579.977868:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189579.978002:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000decf8864 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664371520:12345-0@lo:38:kworker/u8:0.0 Request processed in 145us (10125us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189579.978009:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57483 00000100:00100000:0.0:1776189579.978328:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8670 00000100:00100000:0.0:1776189579.978334:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007ca965f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664371840:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189579.978376:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007ca965f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664371840:12345-0@lo:400:kworker/u8:0.0 Request processed in 42us (11294us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189579.978382:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8670 00000100:00100000:0.0:1776189579.978395:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8671 00000100:00100000:0.0:1776189579.978398:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008be8e55e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664371648:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189579.978458:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008be8e55e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664371648:12345-0@lo:400:kworker/u8:0.0 Request processed in 59us (11126us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189579.978464:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8671 00000100:00100000:0.0:1776189579.978476:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8672 00000100:00100000:0.0:1776189579.978480:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a829435f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664371904:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189579.978508:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a829435f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664371904:12345-0@lo:400:kworker/u8:0.0 Request processed in 28us (10819us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189579.978514:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8672 00000100:00100000:0.0:1776189579.978546:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664371584 00000100:00100000:0.0:1776189579.978579:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9160 00000100:00100000:0.0:1776189579.978583:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d45cc7cd pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664371584:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189579.978591:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189579.978617:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d45cc7cd pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664371584:12345-0@lo:38:kworker/u8:0.0 Request processed in 33us (10718us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189579.978623:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9160 00000100:00100000:0.0:1776189579.978755:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a7b82f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664371968:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189579.978781:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017cf884a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664371200:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189579.978788:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d92ffa01 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664371648:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189579.978793:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b827fed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664371904:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189579.978799:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000526ed979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664371840:0@lo:400:kworker/u8:0.0 00000100:00080000:0.0:1776189579.978826:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189579.978828:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189579.978830:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02fb24b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664371328:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776189579.978836:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189579.978838:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189579.978839:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000053ba7e65 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664371520:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776189579.978844:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189579.978846:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189579.978847:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b7a82fb0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664371584:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189579.982783:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000022f2aa27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664371776:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189579.982809:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000067c9d69b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664371712:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189579.982819:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bf60fbf3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664371264:0@lo:400:kworker/u8:0.0 00010000:02020000:0.0:1776189579.996282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189580.252281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189580.287718:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189580.509928:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189580.764342:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189581.020440:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189581.276279:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189581.308809:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189581.532630:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189581.789875:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189582.044306:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189582.302082:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189582.332379:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189582.556299:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189582.812418:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189583.070472:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189583.325280:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189583.358009:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189583.580314:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189583.738272:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189583.738321:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189583.738357:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189583.738444:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654961024 00000100:00100000:3.0:1776189583.738508:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654961088 00000100:00100000:3.0:1776189583.738527:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10108 00000100:00100000:3.0:1776189583.738534:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001706ae06 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654961024:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1776189583.738623:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001706ae06 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654961024:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 89us (358us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189583.738634:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10108 00000100:00100000:3.0:1776189583.738652:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57484 00000100:00100000:3.0:1776189583.738655:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cab21754 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654961088:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:3.0:1776189583.738669:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776189583.738695:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cab21754 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654961088:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 39us (340us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189583.738701:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57484 00000100:00100000:0.0:1776189583.739460:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654960960 00000100:00100000:0.0:1776189583.739506:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1532 00000100:00100000:0.0:1776189583.739511:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ef3ea18a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654960960:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1776189583.739557:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ef3ea18a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654960960:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 45us (1235us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189583.739570:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1532 00010000:02020000:0.0:1776189583.836287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189584.060496:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007bf8226c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664372032:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189584.060562:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189584.060709:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004447f539 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664372096:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189584.060738:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189584.060801:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664372032 00000100:00100000:2.0:1776189584.060853:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664372096 00000100:00100000:2.0:1776189584.060872:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6948 00000100:00100000:2.0:1776189584.060877:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009b827fed pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664372032:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189584.060906:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189584.060947:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009b827fed pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664372032:12345-0@lo:13:osp-pre-1-0.0 Request processed in 69us (386us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189584.060954:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6948 00000100:00100000:2.0:1776189584.060969:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6949 00000100:00100000:2.0:1776189584.060972:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009ea5366f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664372096:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189584.060987:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189584.061083:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007bf8226c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664372032:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189584.064502:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004447f539 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664372096:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189584.064519:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009ea5366f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664372096:12345-0@lo:13:osp-pre-0-0.0 Request processed in 3542us (3777us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189584.064531:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6949 00010000:02020000:0.0:1776189584.092332:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189584.348382:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189584.380394:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189584.604263:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189584.861267:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189585.084503:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c6bc9cdf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664372352:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189585.084570:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e39f5436 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664372160:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189585.084587:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189585.084635:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664372352 00000100:00100000:1.0:1776189585.084645:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007bf8226c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664372288:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189585.084675:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6950 00000100:00100000:1.0:1776189585.084682:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189585.084685:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d4382e95 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664372352:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189585.084709:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189585.084710:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189585.084749:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664372160 00000100:00100000:3.0:1776189585.084759:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d4382e95 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664372352:12345-0@lo:13:osp-pre-1-1.0 Request processed in 77us (173us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189585.084767:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6950 00000100:00100000:3.0:1776189585.084783:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664372288 00000100:00100000:3.0:1776189585.084814:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6951 00000100:00100000:3.0:1776189585.084818:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000960dca04 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664372288:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776189585.084827:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00080000:0.0:1776189585.084981:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189585.084987:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189585.084990:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189585.084995:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9726 00000100:00080000:0.0:1776189585.084997:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00080000:0.0:1776189585.085001:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189585.085021:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000087b82cd7 x1862459664372416/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189585.085045:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189585.085048:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189585.085049:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189585.085052:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9726 00000100:00080000:0.0:1776189585.085054:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00080000:0.0:1776189585.085057:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189585.085068:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000023121a7 x1862459664372544/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189585.085075:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189585.085077:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189585.085078:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189585.085081:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9726 00000100:00080000:0.0:1776189585.085082:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 6s 00000100:00080000:0.0:1776189585.085085:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189585.085093:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004be216e7 x1862459664372608/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189585.085127:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000087b82cd7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664372416:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189585.085155:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189585.085170:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000023121a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664372544:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189585.085211:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189585.085225:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004be216e7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664372608:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189585.085246:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189585.085274:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664372416 00000100:00100000:0.0:1776189585.085293:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664372544 00000100:00100000:0.0:1776189585.085307:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57485 00000100:00100000:0.0:1776189585.085311:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000069c2fab0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664372416:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189585.085325:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189585.085355:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000069c2fab0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664372416:12345-0@lo:38:kworker/u8:1.0 Request processed in 43us (199us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189585.085361:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57485 00000100:00100000:0.0:1776189585.085372:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57486 00000100:00100000:0.0:1776189585.085375:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000fce6caf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664372544:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189585.085382:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189585.085403:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000fce6caf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664372544:12345-0@lo:38:kworker/u8:1.0 Request processed in 26us (191us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189585.085408:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57486 00000100:00100000:0.0:1776189585.085424:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664372608 00000100:00100000:0.0:1776189585.085441:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9161 00000100:00100000:0.0:1776189585.085444:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005788f021 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664372608:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189585.085451:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189585.085473:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005788f021 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664372608:12345-0@lo:38:kworker/u8:1.0 Request processed in 28us (227us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189585.085479:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9161 00000100:00080000:0.0:1776189585.085527:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189585.085531:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189585.085535:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000087b82cd7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664372416:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776189585.085553:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189585.085555:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189585.085558:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000023121a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664372544:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776189585.085567:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189585.085570:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189585.085573:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004be216e7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664372608:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189585.085596:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005f6664e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664372480:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189585.085617:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189585.085631:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776189585.085637:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d97d7d85 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664372800:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189585.085655:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189585.085676:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664372480 00000100:00100000:0.0:1776189585.085701:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664372800 00000100:00100000:0.0:1776189585.085717:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10109 00000100:00100000:0.0:1776189585.085721:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004be216e7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664372480:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189585.085729:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1533 00000100:00100000:2.0:1776189585.085736:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005c923f64 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664372160:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189585.085757:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004be216e7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664372480:12345-0@lo:400:kworker/u8:1.0 Request processed in 35us (139us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189585.085764:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10109 00000100:00100000:0.0:1776189585.085777:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10110 00000100:00100000:0.0:1776189585.085780:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000023121a7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664372800:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189585.086385:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000023121a7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664372800:12345-0@lo:400:kworker/u8:1.0 Request processed in 603us (728us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189585.086394:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10110 00000100:00100000:2.0:1776189585.086403:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005c923f64 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664372160:12345-0@lo:400:kworker/u8:1.0 Request processed in 666us (1693us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189585.086412:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1533 00000100:00100000:0.0:1776189585.086441:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d97d7d85 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664372800:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189585.086455:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f6664e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664372480:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189585.086463:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e39f5436 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664372160:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189585.086493:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017cf884a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664372224:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189585.086503:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c6bc9cdf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664372352:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189585.086520:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189585.086552:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664372224 00000100:00100000:0.0:1776189585.086574:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10111 00000100:00100000:0.0:1776189585.086577:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e39f5436 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664372224:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189585.086605:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e39f5436 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664372224:12345-0@lo:400:kworker/u8:1.0 Request processed in 27us (86us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189585.086610:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10111 00000100:00100000:0.0:1776189585.086628:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602ee85e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664372672:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189585.086651:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189585.086675:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017cf884a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664372224:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189585.086684:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776189585.086690:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007df90c3e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664372736:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189585.086710:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189585.086871:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664372672 00000100:00100000:0.0:1776189585.086906:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664372736 00000100:00100000:0.0:1776189585.086928:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8673 00000100:00100000:0.0:1776189585.086932:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d97d7d85 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664372672:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189585.086961:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d97d7d85 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664372672:12345-0@lo:400:kworker/u8:1.0 Request processed in 29us (311us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189585.086967:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8673 00000100:00100000:0.0:1776189585.086977:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8674 00000100:00100000:0.0:1776189585.086981:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000017cf884a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664372736:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189585.087002:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000017cf884a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664372736:12345-0@lo:400:kworker/u8:1.0 Request processed in 21us (292us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189585.087007:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8674 00000100:00100000:0.0:1776189585.087394:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602ee85e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664372672:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189585.087409:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007df90c3e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664372736:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189585.097244:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000960dca04 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664372288:12345-0@lo:13:osp-pre-0-1.0 Request processed in 12415us (12554us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189585.097262:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6951 00000100:00100000:1.0:1776189585.097873:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007bf8226c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664372288:0@lo:13:osp-pre-0-1.0 00010000:02020000:0.0:1776189585.116294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189585.372708:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189585.404423:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189585.629868:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189585.884300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189586.140840:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189586.396289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189586.428511:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189586.652463:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189586.911285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189587.164853:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189587.420366:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189587.454602:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189587.676293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189587.936027:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189588.188399:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189588.445920:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189588.486470:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189588.702932:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189588.860262:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189588.860360:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189588.860406:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189588.860487:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654961216 00000100:00100000:0.0:1776189588.860541:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654961280 00000100:00100000:0.0:1776189588.860560:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10112 00000100:00100000:0.0:1776189588.860566:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d01df670 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654961216:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189588.860643:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d01df670 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654961216:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 76us (283us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189588.860650:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10112 00000100:00100000:0.0:1776189588.860667:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57487 00000100:00100000:0.0:1776189588.860670:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000502b3e4a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654961280:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776189588.860683:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189588.860724:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000502b3e4a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654961280:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 52us (319us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189588.860730:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57487 00000100:00100000:2.0:1776189588.865883:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654961152 00000100:00100000:2.0:1776189588.865957:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1534 00000100:00100000:2.0:1776189588.865964:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cd00b680 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654961152:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189588.866221:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cd00b680 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654961152:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 256us (5967us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189588.866229:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1534 00010000:02020000:0.0:1776189588.956300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189589.180521:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776189589.180548:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a0b45fb4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664372864:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189589.180618:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189589.180740:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664372864 00000100:00100000:3.0:1776189589.180790:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6952 00000100:00100000:3.0:1776189589.180796:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd5e571c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664372864:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776189589.180824:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189589.180886:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd5e571c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664372864:12345-0@lo:13:osp-pre-1-0.0 Request processed in 89us (268us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189589.180893:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6952 00000100:00100000:1.0:1776189589.180981:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a0b45fb4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664372864:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189589.181713:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776189589.181725:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a943cb24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664372928:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189589.181760:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189589.181832:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664372928 00000100:00100000:2.0:1776189589.181880:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6953 00000100:00100000:2.0:1776189589.181885:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0b45fb4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664372928:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189589.181904:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189589.181946:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0b45fb4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664372928:12345-0@lo:13:osp-pre-0-0.0 Request processed in 60us (187us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189589.181953:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6953 00000100:00100000:1.0:1776189589.182098:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a943cb24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664372928:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189589.212316:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189589.469066:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189589.500534:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189589.724761:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189589.980285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:3.0:1776189590.204877:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189590.204885:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189590.204888:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189590.204898:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9732 00000100:00080000:3.0:1776189590.204902:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00080000:3.0:1776189590.204908:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189590.204911:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776189590.204929:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004447f539 x1862459664373184/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189590.204943:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a943cb24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664372992:0@lo:400:kworker/u8:1.0 00000100:00080000:3.0:1776189590.204954:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189590.204958:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189590.204959:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189590.204963:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9732 00000100:00080000:3.0:1776189590.204965:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:3.0:1776189590.204968:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189590.204981:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ee179fa1 x1862459664373312/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189590.204990:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189590.204993:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189590.204995:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189590.204998:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9732 00000100:00080000:3.0:1776189590.205000:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 11s 00000100:00080000:3.0:1776189590.205003:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189590.205014:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000058863251 x1862459664373376/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189590.205020:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189590.205051:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a7b82f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664373248:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189590.205084:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007df90c3e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664373120:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189590.205085:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189590.205108:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f8dcf939 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664373632:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189590.205125:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189590.205141:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189590.205148:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f40573b9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664373568:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189590.205164:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1776189590.205191:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d52de5d9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664373440:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189590.205199:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189590.205221:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189590.205221:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1776189590.205229:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c6bc9cdf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664373056:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189590.205243:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000838deece pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664373696:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189590.205259:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189590.205274:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189590.205283:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005c6d79ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664373504:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189590.205311:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664373632 00000100:00100000:3.0:1776189590.205312:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189590.205343:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004447f539 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664373184:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189590.205353:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664373440 00000100:00100000:3.0:1776189590.205370:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189590.205374:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664373696 00000100:00100000:3.0:1776189590.205391:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee179fa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664373312:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189590.205396:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664373504 00000100:00100000:3.0:1776189590.205422:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189590.205423:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8675 00000100:00100000:0.0:1776189590.205431:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f6664e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664373632:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189590.205441:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000058863251 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664373376:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189590.205464:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189590.205496:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664373248 00000100:00100000:3.0:1776189590.205525:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664373568 00000100:00100000:3.0:1776189590.205537:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664373056 00000100:00100000:3.0:1776189590.205547:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664373184 00000100:00100000:3.0:1776189590.205554:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664373312 00000100:00100000:3.0:1776189590.205565:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10113 00000100:00100000:3.0:1776189590.205568:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000602ee85e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664373248:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189590.205599:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000602ee85e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664373248:12345-0@lo:400:kworker/u8:1.0 Request processed in 29us (517us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189590.205605:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10113 00000100:00100000:3.0:1776189590.205620:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10114 00000100:00100000:3.0:1776189590.205623:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c3d6c201 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664373568:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189590.205653:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c3d6c201 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664373568:12345-0@lo:400:kworker/u8:1.0 Request processed in 29us (456us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189590.205659:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10114 00000100:00100000:3.0:1776189590.205669:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10115 00000100:00100000:3.0:1776189590.205673:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000052e42ae8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664373056:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189590.205709:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000052e42ae8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664373056:12345-0@lo:400:kworker/u8:1.0 Request processed in 35us (452us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189590.205715:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10115 00000100:00100000:3.0:1776189590.205729:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57488 00000100:00100000:3.0:1776189590.205733:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003cd25a35 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664373184:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189590.205743:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664372992 00010000:02000000:3.0:1776189590.205747:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189590.205779:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1535 00000100:00100000:3.0:1776189590.205784:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003cd25a35 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664373184:12345-0@lo:38:kworker/u8:1.0 Request processed in 48us (414us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189590.205786:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f6dc323d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664372992:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189590.205798:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57488 00000100:00100000:3.0:1776189590.205812:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57489 00000100:00100000:3.0:1776189590.205817:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e532ce7c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664373312:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189590.205825:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189590.205830:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f6dc323d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664372992:12345-0@lo:400:kworker/u8:1.0 Request processed in 44us (814us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189590.205838:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1535 00000100:00100000:3.0:1776189590.205856:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e532ce7c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664373312:12345-0@lo:38:kworker/u8:1.0 Request processed in 37us (437us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189590.205863:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57489 00000100:00100000:1.0:1776189590.206004:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000114609f6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664373760:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189590.206033:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189590.206064:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664373120 00000100:00100000:1.0:1776189590.206089:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664373760 00000100:00100000:1.0:1776189590.206102:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6954 00000100:00100000:1.0:1776189590.206106:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000038a7e045 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3362:x1862459664373120:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189590.206119:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189590.206148:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000038a7e045 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3362:x1862459664373120:12345-0@lo:13:osp-pre-1-1.0 Request processed in 41us (1025us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189590.206154:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6954 00000100:00100000:1.0:1776189590.206168:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6955 00000100:00100000:1.0:1776189590.206186:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001ce461df pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3361:x1862459664373760:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189590.206197:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189590.206224:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001ce461df pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3361:x1862459664373760:12345-0@lo:13:osp-pre-0-1.0 Request processed in 52us (192us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189590.206230:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6955 00000100:00100000:1.0:1776189590.206263:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000114609f6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664373760:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189590.206301:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8676 00000100:00100000:1.0:1776189590.206306:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000087b82cd7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664373440:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189590.206334:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000087b82cd7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664373440:12345-0@lo:400:kworker/u8:1.0 Request processed in 28us (1115us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189590.206340:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8676 00000100:00100000:1.0:1776189590.206352:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8677 00000100:00100000:1.0:1776189590.206356:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000db568034 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664373696:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189590.206379:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000db568034 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664373696:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (1108us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189590.206384:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8677 00000100:00100000:1.0:1776189590.206393:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8678 00000100:00100000:1.0:1776189590.206397:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000068da6036 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664373504:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189590.206418:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000068da6036 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664373504:12345-0@lo:400:kworker/u8:1.0 Request processed in 21us (1107us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189590.206424:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8678 00000100:00100000:1.0:1776189590.206495:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d52de5d9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664373440:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189590.206516:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000838deece pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664373696:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189590.206525:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f8dcf939 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664373632:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189590.206533:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a7b82f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664373248:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189590.206541:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a943cb24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664372992:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189590.206609:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f6664e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664373632:12345-0@lo:400:kworker/u8:1.0 Request processed in 71us (362us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189590.206617:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8675 00000100:00100000:3.0:1776189590.223957:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664373376 00000100:00100000:3.0:1776189590.225209:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9162 00000100:00100000:3.0:1776189590.225217:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c68cc49e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664373376:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189590.225235:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189590.225311:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c68cc49e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664373376:12345-0@lo:38:kworker/u8:1.0 Request processed in 94us (19846us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189590.225319:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9162 00000100:00080000:3.0:1776189590.225367:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189590.225371:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189590.225375:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004447f539 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664373184:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776189590.225384:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189590.225387:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189590.225389:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee179fa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664373312:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776189590.225397:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189590.225400:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189590.225402:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000058863251 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664373376:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189590.225451:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c6bc9cdf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664373056:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189590.225462:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005c6d79ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664373504:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189590.225478:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007df90c3e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664373120:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189590.225490:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f40573b9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664373568:0@lo:400:kworker/u8:1.0 00010000:02020000:2.0:1776189590.236405:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189590.492282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189590.524369:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189590.748284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189591.004830:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189591.260288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189591.516287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189591.549148:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189591.773095:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189592.031890:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189592.287240:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189592.540379:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189592.576995:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189592.796285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189593.052385:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189593.308427:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189593.564278:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189593.596469:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189593.820308:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189593.980032:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189593.980093:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189593.980145:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189593.980217:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654961472 00000100:00100000:0.0:1776189593.980270:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654961408 00000100:00100000:3.0:1776189593.980290:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10116 00000100:00100000:0.0:1776189593.980295:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57490 00000100:00100000:3.0:1776189593.980297:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c04acb9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654961408:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189593.980299:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f40573b9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654961472:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776189593.980327:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776189593.980361:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c04acb9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654961408:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 64us (269us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189593.980369:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10116 00000100:00100000:1.0:1776189593.980388:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654961344 00000100:00100000:1.0:1776189593.980415:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1536 00000100:00100000:1.0:1776189593.980420:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000078287bc4 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654961344:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189593.990866:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f40573b9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654961472:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 10554us (10835us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189593.990885:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57490 00000100:00100000:1.0:1776189593.991288:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000078287bc4 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654961344:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 10864us (11141us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189593.991297:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1536 00010000:02020000:2.0:1776189594.076373:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189594.302513:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776189594.302535:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000002c44196 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664373824:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189594.302609:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189594.302693:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664373824 00000100:00100000:0.0:1776189594.302733:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6956 00000100:00100000:0.0:1776189594.302739:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000114609f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664373824:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189594.302767:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189594.302979:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000002c44196 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664373824:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189594.303014:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000114609f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664373824:12345-0@lo:13:osp-pre-1-0.0 Request processed in 90us (221us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189594.303022:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6956 00000100:00100000:1.0:1776189594.303649:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f8dcf939 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664373888:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189594.303681:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189594.303875:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664373888 00000100:00100000:3.0:1776189594.304620:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6957 00000100:00100000:3.0:1776189594.304628:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005c6d79ae pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664373888:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189594.304654:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189594.304716:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005c6d79ae pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664373888:12345-0@lo:13:osp-pre-0-0.0 Request processed in 88us (1034us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189594.304722:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6957 00000100:00100000:2.0:1776189594.305466:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f8dcf939 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664373888:0@lo:13:osp-pre-0-0.0 00010000:02020000:2.0:1776189594.332318:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189594.588625:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189594.623437:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189594.844392:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189595.103222:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189595.324598:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c6bc9cdf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664373952:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189595.324667:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189595.324783:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664373952 00000100:00100000:1.0:1776189595.324834:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6958 00000100:00100000:1.0:1776189595.324840:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cf6b5d65 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664373952:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189595.324863:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189595.324911:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cf6b5d65 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664373952:12345-0@lo:13:osp-pre-0-1.0 Request processed in 70us (245us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189595.324917:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6958 00000100:00100000:0.0:1776189595.325165:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c6bc9cdf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664373952:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189595.325603:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000002c44196 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664374016:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189595.325640:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189595.325726:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664374016 00000100:00080000:1.0:1776189595.325929:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189595.325934:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189595.325937:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189595.325942:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9737 00000100:00080000:1.0:1776189595.325945:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:1.0:1776189595.325949:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189595.325968:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004447f539 x1862459664374208/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189595.325991:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189595.325994:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189595.325995:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189595.325998:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9737 00000100:00080000:1.0:1776189595.326000:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00080000:1.0:1776189595.326003:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189595.326011:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005a7b82f1 x1862459664374336/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189595.326018:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189595.326021:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189595.326022:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189595.326025:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9737 00000100:00080000:1.0:1776189595.326027:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00080000:1.0:1776189595.326029:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189595.326040:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d52de5d9 x1862459664374400/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189595.326076:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee179fa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664374144:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189595.326107:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189595.326123:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad3c7e22 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664374592:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189595.326142:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189595.326156:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776189595.326160:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a943cb24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664374272:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189595.326657:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6959 00000100:00100000:3.0:1776189595.326663:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004dd53b72 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664374016:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189595.326682:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189595.326719:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004dd53b72 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664374016:12345-0@lo:13:osp-pre-1-1.0 Request processed in 55us (1079us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189595.326725:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6959 00000100:00100000:1.0:1776189595.326775:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189595.326809:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664374144 00000100:00100000:1.0:1776189595.326841:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664374592 00000100:00100000:1.0:1776189595.326859:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664374272 00000100:00100000:1.0:1776189595.326875:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10117 00000100:00100000:1.0:1776189595.326884:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000086fda571 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664374144:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189595.326920:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000086fda571 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664374144:12345-0@lo:400:kworker/u8:0.0 Request processed in 35us (813us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189595.326925:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10117 00000100:00100000:1.0:1776189595.326937:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10118 00000100:00100000:1.0:1776189595.326941:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022af59eb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664374592:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189595.326965:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022af59eb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664374592:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (823us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189595.326970:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10118 00000100:00100000:1.0:1776189595.326979:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10119 00000100:00100000:1.0:1776189595.326982:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f6d90a77 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664374272:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189595.327003:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f6d90a77 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664374272:12345-0@lo:400:kworker/u8:0.0 Request processed in 19us (229us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189595.327008:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10119 00000100:00100000:1.0:1776189595.327040:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a943cb24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664374272:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189595.327050:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee179fa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664374144:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189595.327058:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad3c7e22 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664374592:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189595.327114:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004447f539 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664374208:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189595.327139:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189595.327152:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a7b82f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664374336:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189595.327168:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189595.327226:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d52de5d9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664374400:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189595.327249:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189595.327273:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664374208 00000100:00100000:1.0:1776189595.327289:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664374336 00000100:00100000:1.0:1776189595.327298:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57491 00000100:00100000:1.0:1776189595.327301:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009f0944ec pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664374208:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189595.327314:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189595.327342:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009f0944ec pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664374208:12345-0@lo:38:kworker/u8:0.0 Request processed in 40us (203us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189595.327348:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57491 00000100:00100000:1.0:1776189595.327359:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57492 00000100:00100000:1.0:1776189595.327362:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ad3c7e22 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664374336:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189595.327368:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189595.327386:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ad3c7e22 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664374336:12345-0@lo:38:kworker/u8:0.0 Request processed in 23us (218us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189595.327391:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57492 00000100:00080000:1.0:1776189595.327414:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189595.327416:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189595.327419:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004447f539 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664374208:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776189595.327426:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189595.327428:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189595.327430:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a7b82f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664374336:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189595.332131:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664374400 00000100:00100000:2.0:1776189595.332248:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9163 00000100:00100000:2.0:1776189595.332254:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ee179fa1 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664374400:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189595.332273:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189595.332354:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ee179fa1 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664374400:12345-0@lo:38:kworker/u8:0.0 Request processed in 99us (5105us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189595.332361:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9163 00000100:00100000:2.0:1776189595.332390:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000058863251 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664374080:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189595.332430:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189595.332451:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007bf8226c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664374528:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189595.332487:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189595.332508:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776189595.332514:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c6bc9cdf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664374464:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189595.332536:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189595.332576:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664374528 00000100:00100000:2.0:1776189595.332605:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664374464 00000100:00100000:2.0:1776189595.332623:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8679 00000100:00100000:2.0:1776189595.332627:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005d3cfd23 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664374528:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189595.332668:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005d3cfd23 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664374528:12345-0@lo:400:kworker/u8:0.0 Request processed in 40us (181us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189595.332673:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8679 00000100:00100000:2.0:1776189595.332686:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8680 00000100:00100000:2.0:1776189595.332689:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bf60fbf3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664374464:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189595.332713:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bf60fbf3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664374464:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (177us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189595.332718:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8680 00000100:00100000:2.0:1776189595.332743:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664374080 00000100:00100000:2.0:1776189595.332777:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1537 00000100:00100000:2.0:1776189595.332780:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003afac4ae pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664374080:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189595.332807:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003afac4ae pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664374080:12345-0@lo:400:kworker/u8:0.0 Request processed in 26us (377us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189595.332816:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1537 00000100:00080000:2.0:1776189595.332863:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189595.332867:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189595.332870:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d52de5d9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664374400:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189595.332901:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c6bc9cdf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664374464:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189595.332908:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000058863251 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664374080:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189595.332915:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007bf8226c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664374528:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189595.332954:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000002c44196 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664374016:0@lo:13:osp-pre-1-1.0 00010000:02020000:0.0:1776189595.359314:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189595.612330:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189595.650986:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189595.868287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189596.124309:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189596.380278:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189596.636484:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189596.670397:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189596.894752:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189597.148549:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189597.408435:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189597.660667:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189597.693276:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189597.916304:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189598.173932:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189598.428411:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189598.684625:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189598.716451:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189598.940425:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189599.098449:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189599.098508:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189599.098596:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654961600 00000100:00100000:2.0:1776189599.098667:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10120 00000100:00100000:2.0:1776189599.098674:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005a7b82f1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654961600:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776189599.098743:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005a7b82f1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654961600:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 69us (301us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189599.098751:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10120 00000100:00100000:0.0:1776189599.098826:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654961536 00000100:00100000:0.0:1776189599.098863:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1538 00000100:00100000:0.0:1776189599.098869:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000921cb3a0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654961536:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1776189599.098912:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000921cb3a0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654961536:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 43us (403us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189599.098918:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1538 00000100:00100000:1.0:1776189599.100121:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189599.100263:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654961664 00000100:00100000:2.0:1776189599.100300:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57493 00000100:00100000:2.0:1776189599.100305:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004447f539 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654961664:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:2.0:1776189599.100323:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189599.100384:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004447f539 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654961664:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 78us (269us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189599.100397:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57493 00010000:02020000:0.0:1776189599.196587:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189599.420898:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000838deece pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664374656:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189599.420965:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189599.421136:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664374656 00000100:00100000:3.0:1776189599.421211:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6960 00000100:00100000:3.0:1776189599.421216:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a943cb24 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664374656:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189599.421245:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189599.421551:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a943cb24 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664374656:12345-0@lo:13:osp-pre-0-0.0 Request processed in 333us (586us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189599.421559:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6960 00000100:00100000:1.0:1776189599.421721:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776189599.421732:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000002c44196 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664374720:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189599.421764:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189599.421831:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664374720 00000100:00100000:0.0:1776189599.421866:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6961 00000100:00100000:0.0:1776189599.421869:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000058863251 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664374720:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189599.421887:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189599.421924:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000058863251 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664374720:12345-0@lo:13:osp-pre-1-0.0 Request processed in 53us (159us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189599.421930:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6961 00000100:00100000:0.0:1776189599.421965:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000002c44196 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664374720:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189599.423525:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000838deece pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664374656:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189599.453876:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189599.711890:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189599.744321:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189599.964316:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189600.220295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189600.444354:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776189600.444379:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000067c9d69b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664374848:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189600.444384:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c6bc9cdf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664374784:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189600.444495:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000838deece pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664374912:0@lo:400:kworker/u8:0.0 00000100:00080000:3.0:1776189600.444498:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189600.444505:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189600.444508:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189600.444513:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9742 00000100:00080000:3.0:1776189600.444516:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189600.444522:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189600.444541:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007df90c3e x1862459664375040/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189600.444565:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189600.444568:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189600.444570:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189600.444573:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9742 00000100:00080000:3.0:1776189600.444575:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189600.444578:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189600.444587:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000071348c99 x1862459664375168/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189600.444595:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189600.444597:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189600.444599:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189600.444602:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9742 00000100:00080000:3.0:1776189600.444604:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189600.444607:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189600.444644:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000611a3c66 x1862459664375232/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189600.444693:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3185a52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664375104:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189600.447094:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189600.447162:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f8dcf939 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664374976:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189600.447246:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189600.447266:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bb8567a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664375424:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189600.447287:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189600.447331:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664374784 00000100:00100000:1.0:1776189600.447545:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6962 00000100:00100000:1.0:1776189600.447551:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d52de5d9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664374784:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189600.447574:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189600.447610:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d52de5d9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664374784:12345-0@lo:13:osp-pre-0-1.0 Request processed in 59us (524us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189600.447616:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6962 00000100:00100000:1.0:1776189600.447845:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664374976 00000100:00100000:1.0:1776189600.447873:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664375424 00000100:00100000:1.0:1776189600.447888:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10121 00000100:00100000:1.0:1776189600.447892:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000aa4e2605 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664374976:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189600.447924:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000aa4e2605 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664374976:12345-0@lo:400:kworker/u8:0.0 Request processed in 31us (679us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189600.447933:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10121 00000100:00100000:1.0:1776189600.447945:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10122 00000100:00100000:1.0:1776189600.447948:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000db600834 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664375424:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189600.447972:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000db600834 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664375424:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (684us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189600.447977:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10122 00000100:00100000:1.0:1776189600.448029:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f8dcf939 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664374976:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189600.448041:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bb8567a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664375424:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189600.448055:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c6bc9cdf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664374784:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189600.448113:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007df90c3e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664375040:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189600.448140:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189600.448154:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000071348c99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664375168:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189600.448195:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189600.448207:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000611a3c66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664375232:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189600.449521:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189600.454727:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ac2fe2ac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664375360:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189600.454829:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189600.454867:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f1ff984b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664375296:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189600.454897:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189600.454915:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000058fbcadb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664375552:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189600.454936:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189600.454993:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664374912 00000100:00100000:2.0:1776189600.455040:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1539 00000100:00100000:2.0:1776189600.455046:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008da73e5e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664374912:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189600.455092:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008da73e5e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664374912:12345-0@lo:400:kworker/u8:0.0 Request processed in 46us (5575us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189600.455099:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1539 00000100:00100000:2.0:1776189600.455133:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664375296 00000100:00100000:2.0:1776189600.455155:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664375552 00000100:00100000:2.0:1776189600.455188:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8681 00000100:00100000:2.0:1776189600.455192:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b7a82fb0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664375296:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189600.455227:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b7a82fb0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664375296:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (329us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189600.455232:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8681 00000100:00100000:2.0:1776189600.455243:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8682 00000100:00100000:2.0:1776189600.455247:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053ba7e65 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664375552:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189600.455272:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053ba7e65 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664375552:12345-0@lo:400:kworker/u8:0.0 Request processed in 25us (337us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189600.455281:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8682 00000100:00100000:2.0:1776189600.455306:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664374848 00000100:00100000:2.0:1776189600.455327:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6963 00000100:00100000:2.0:1776189600.455330:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022f2aa27 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664374848:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189600.455344:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189600.455371:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022f2aa27 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664374848:12345-0@lo:13:osp-pre-1-1.0 Request processed in 39us (543us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189600.455376:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6963 00000100:00100000:2.0:1776189600.455415:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f1ff984b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664375296:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189600.455424:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000058fbcadb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664375552:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189600.455438:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000067c9d69b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664374848:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189600.455517:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664375040 00000100:00100000:2.0:1776189600.455536:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664375168 00000100:00100000:2.0:1776189600.455550:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57494 00000100:00100000:2.0:1776189600.455553:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002ac4dbc5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664375040:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189600.455567:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189600.456636:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189600.456643:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189600.456679:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000eeb8de0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664375488:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189600.456707:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189600.456754:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664375488 00000100:00100000:1.0:1776189600.456764:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664375104 00000100:00100000:3.0:1776189600.456790:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8683 00000100:00100000:3.0:1776189600.456797:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a9575a34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664375488:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189600.456801:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10123 00000100:00100000:1.0:1776189600.456808:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a5008497 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664375104:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189600.456843:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a9575a34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664375488:12345-0@lo:400:kworker/u8:0.0 Request processed in 47us (136us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189600.456851:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8683 00000100:00100000:3.0:1776189600.456886:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000eeb8de0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664375488:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189600.456912:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a5008497 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664375104:12345-0@lo:400:kworker/u8:0.0 Request processed in 102us (269us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189600.456912:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3185a52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664375104:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189600.456925:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10123 00000100:00100000:1.0:1776189600.456942:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57495 00000100:00100000:1.0:1776189600.456946:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000da0d27b2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664375168:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189600.456959:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189600.456975:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002ac4dbc5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664375040:12345-0@lo:38:kworker/u8:0.0 Request processed in 1418us (8833us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189600.457050:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57494 00000100:00100000:2.0:1776189600.457099:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664375232 00000100:00100000:2.0:1776189600.457121:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9164 00000100:00100000:1.0:1776189600.457128:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000da0d27b2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664375168:12345-0@lo:38:kworker/u8:0.0 Request processed in 177us (8929us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189600.457128:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000431c2394 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664375232:12345-0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189600.457143:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57495 00010000:02000000:2.0:1776189600.457144:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189600.457202:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000431c2394 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664375232:12345-0@lo:38:kworker/u8:0.0 Request processed in 76us (570us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189600.457209:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9164 00000100:00080000:1.0:1776189600.457478:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189600.457482:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189600.457487:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007df90c3e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664375040:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776189600.457497:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189600.457500:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189600.457502:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000071348c99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664375168:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776189600.457511:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189600.457513:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189600.457515:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000611a3c66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664375232:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189600.463164:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189600.463245:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000838deece pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664374912:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189600.463892:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664375360 00000100:00100000:3.0:1776189600.463953:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8684 00000100:00100000:3.0:1776189600.463959:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000002c44196 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664375360:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189600.464031:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000002c44196 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664375360:12345-0@lo:400:kworker/u8:0.0 Request processed in 71us (874us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189600.464040:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8684 00000100:00100000:0.0:1776189600.464255:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ac2fe2ac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664375360:0@lo:400:kworker/u8:0.0 00010000:02020000:0.0:1776189600.476339:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189600.732301:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189600.765760:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189600.990939:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189601.244288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189601.500350:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189601.756301:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189601.789971:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189602.012274:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189602.269041:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189602.525526:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189602.780369:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189602.815203:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189603.036429:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189603.292294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189603.548307:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189603.804423:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189603.839803:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189604.060461:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189604.224579:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189604.224621:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189604.224729:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189604.225001:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654961728 00000100:00100000:2.0:1776189604.225061:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1540 00000100:00100000:2.0:1776189604.225067:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c3d39af pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654961728:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189604.225087:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654961856 00000100:00100000:1.0:1776189604.225109:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654961792 00000100:00100000:1.0:1776189604.225129:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10124 00000100:00100000:1.0:1776189604.225133:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ef812603 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654961792:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189604.225134:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c3d39af pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654961728:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 66us (405us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189604.225141:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1540 00000100:00100000:1.0:1776189604.225206:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ef812603 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654961792:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 71us (584us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189604.225213:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10124 00000100:00100000:1.0:1776189604.225227:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57496 00000100:00100000:1.0:1776189604.225230:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000eeb8de0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654961856:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776189604.225244:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189604.225268:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000eeb8de0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654961856:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 37us (698us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189604.225274:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57496 00010000:02020000:0.0:1776189604.318595:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189604.540825:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776189604.540849:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000603f73df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664375616:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189604.540928:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189604.541059:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664375616 00000100:00100000:3.0:1776189604.541119:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6964 00000100:00100000:3.0:1776189604.541124:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000067c9d69b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664375616:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189604.541158:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189604.541257:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000067c9d69b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664375616:12345-0@lo:13:osp-pre-0-0.0 Request processed in 131us (329us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189604.541264:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6964 00000100:00100000:1.0:1776189604.541505:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000603f73df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664375616:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189604.542426:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776189604.542441:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000545eaa39 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664375680:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189604.542489:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189604.542572:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664375680 00000100:00100000:1.0:1776189604.542613:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6965 00000100:00100000:1.0:1776189604.542617:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006e96e3d4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664375680:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189604.542639:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189604.542687:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006e96e3d4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664375680:12345-0@lo:13:osp-pre-1-0.0 Request processed in 69us (199us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189604.542694:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6965 00000100:00100000:0.0:1776189604.542776:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000545eaa39 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664375680:0@lo:13:osp-pre-1-0.0 00010000:02020000:0.0:1776189604.572933:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189604.828446:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189604.861784:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189605.084282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189605.340392:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189605.564585:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776189605.564611:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000054387670 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664375744:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189605.564692:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189605.564887:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664375744 00000100:00100000:0.0:1776189605.565089:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6966 00000100:00100000:0.0:1776189605.565096:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008821baff pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664375744:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776189605.565122:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189605.565198:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008821baff pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664375744:12345-0@lo:13:osp-pre-0-1.0 Request processed in 102us (507us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189605.565207:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6966 00000100:00100000:3.0:1776189605.565372:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000054387670 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664375744:0@lo:13:osp-pre-0-1.0 00000100:00080000:2.0:1776189605.565553:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189605.565558:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189605.565561:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189605.565567:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9747 00000100:00080000:2.0:1776189605.565569:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776189605.565573:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189605.565592:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d92ffa01 x1862459664375936/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189605.565619:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189605.565623:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189605.565624:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189605.565627:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9747 00000100:00080000:2.0:1776189605.565629:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776189605.565632:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189605.565740:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009820cd1e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664376000:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189605.565771:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189605.565790:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776189605.565795:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ff5d45b7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664376064:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189605.565817:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189605.565902:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000526ed979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664375872:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189605.565964:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189605.565989:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776189605.565994:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02fb24b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664375808:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189605.566019:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189605.566050:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664375808 00000100:00100000:0.0:1776189605.566080:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1541 00000100:00100000:0.0:1776189605.566084:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e858e44a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664375808:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189605.566119:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e858e44a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664375808:12345-0@lo:400:kworker/u8:1.0 Request processed in 34us (99us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189605.566125:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1541 00000100:00100000:0.0:1776189605.566154:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664376000 00000100:00100000:0.0:1776189605.566196:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664375872 00000100:00100000:0.0:1776189605.566211:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10125 00000100:00100000:0.0:1776189605.566215:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009806e292 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664376000:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189605.566421:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10126 00000100:00100000:3.0:1776189605.566426:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000444946da pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664375872:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189605.566479:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009806e292 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664376000:12345-0@lo:400:kworker/u8:1.0 Request processed in 262us (707us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189605.566485:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10125 00000100:00100000:0.0:1776189605.566518:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02fb24b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664375808:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189605.566696:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007bf8226c x1862459664376128/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189605.566708:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189605.566712:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189605.566713:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189605.566717:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9747 00000100:00080000:2.0:1776189605.566718:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776189605.566721:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189605.566737:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001fe1325f x1862459664376192/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189605.566803:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000048c7c2e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664376384:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189605.566831:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189605.566855:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009820cd1e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664376000:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189605.567114:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664376064 00000100:00100000:1.0:1776189605.567156:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6967 00000100:00100000:1.0:1776189605.567160:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e9138b66 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664376064:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189605.567196:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189605.567248:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e9138b66 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664376064:12345-0@lo:13:osp-pre-1-1.0 Request processed in 87us (1430us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189605.567255:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6967 00000100:00100000:1.0:1776189605.567276:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000023a5b7b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664376256:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189605.567301:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189605.567320:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776189605.567326:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001a975e34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664376320:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189605.567360:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189605.567388:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664376256 00000100:00100000:1.0:1776189605.567412:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664376320 00000100:00100000:1.0:1776189605.567429:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8685 00000100:00100000:1.0:1776189605.567433:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b0552b59 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664376256:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189605.567467:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b0552b59 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664376256:12345-0@lo:400:kworker/u8:1.0 Request processed in 33us (165us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189605.567473:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8685 00000100:00100000:1.0:1776189605.567485:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8686 00000100:00100000:1.0:1776189605.567489:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000314a7eb9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664376320:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189605.567512:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000314a7eb9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664376320:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (152us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189605.567516:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8686 00000100:00100000:1.0:1776189605.567541:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664376384 00000100:00100000:1.0:1776189605.567559:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10127 00000100:00100000:1.0:1776189605.567563:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000005ed8ead pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664376384:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189605.567669:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ff5d45b7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664376064:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189605.567692:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d92ffa01 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664375936:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189605.567718:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189605.567733:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007bf8226c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664376128:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189605.567752:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189605.567763:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001fe1325f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664376192:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189605.567785:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189605.567811:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664376192 00000100:00100000:0.0:1776189605.567833:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9165 00000100:00100000:0.0:1776189605.567837:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001546cd51 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664376192:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189605.567851:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189605.567891:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001546cd51 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664376192:12345-0@lo:38:kworker/u8:1.0 Request processed in 42us (95us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189605.567897:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9165 00000100:00100000:0.0:1776189605.567918:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664375936 00000100:00100000:0.0:1776189605.567944:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664376128 00000100:00100000:0.0:1776189605.567957:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57497 00000100:00100000:0.0:1776189605.567960:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008fcf598b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664375936:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189605.567967:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189605.567989:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008fcf598b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664375936:12345-0@lo:38:kworker/u8:1.0 Request processed in 28us (272us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189605.567995:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57497 00000100:00100000:0.0:1776189605.568004:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57498 00000100:00100000:0.0:1776189605.568007:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006bc6544d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664376128:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189605.568013:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189605.568033:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006bc6544d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664376128:12345-0@lo:38:kworker/u8:1.0 Request processed in 25us (281us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189605.568038:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57498 00000100:00080000:0.0:1776189605.568061:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189605.568066:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189605.568069:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d92ffa01 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664375936:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776189605.568075:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189605.568077:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189605.568080:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007bf8226c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664376128:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776189605.568086:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189605.568088:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189605.568089:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001fe1325f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664376192:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189605.568128:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001a975e34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664376320:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189605.568137:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000023a5b7b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664376256:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189605.571480:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000444946da pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664375872:12345-0@lo:400:kworker/u8:1.0 Request processed in 5046us (5509us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189605.571499:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10126 00000100:00100000:0.0:1776189605.573792:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000526ed979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664375872:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189605.574840:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000005ed8ead pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664376384:12345-0@lo:400:kworker/u8:1.0 Request processed in 7271us (8004us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189605.574852:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10127 00000100:00100000:0.0:1776189605.575474:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000048c7c2e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664376384:0@lo:400:kworker/u8:1.0 00010000:02020000:0.0:1776189605.596296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189605.852413:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189605.884469:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189606.108299:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189606.365061:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189606.620687:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189606.876566:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189606.908607:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189607.132285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189607.388294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189607.644286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189607.900321:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189607.933092:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189608.156373:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189608.412305:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189608.672064:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189608.924353:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189608.958758:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189609.180298:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189609.338671:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189609.338724:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189609.338765:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189609.338799:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654961920 00000100:00100000:3.0:1776189609.338844:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654961984 00000100:00100000:2.0:1776189609.338866:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1542 00000100:00100000:2.0:1776189609.338874:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ff8d0a76 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654961920:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189609.338882:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654962048 00000100:00100000:3.0:1776189609.338902:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10128 00000100:00100000:3.0:1776189609.338906:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ce3ffae5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654961984:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189609.338961:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ff8d0a76 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654961920:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 87us (236us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189609.338969:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1542 00000100:00100000:2.0:1776189609.339015:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57499 00000100:00100000:2.0:1776189609.339019:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000043ca7548 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654962048:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776189609.339034:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776189609.344591:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ce3ffae5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654961984:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 5675us (5920us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189609.344609:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10128 00000100:00100000:2.0:1776189609.346987:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000043ca7548 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654962048:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 7957us (8213us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189609.347006:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57499 00010000:02020000:0.0:1776189609.436299:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189609.660488:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c4b16b97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664376448:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189609.660558:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189609.660845:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664376448 00000100:00100000:2.0:1776189609.660858:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776189609.660869:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005ed146ab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664376512:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189609.660899:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6968 00000100:00100000:2.0:1776189609.660900:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189609.660909:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e9a35d2f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664376448:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189609.660940:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189609.660941:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664376512 00000100:00100000:2.0:1776189609.660967:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6969 00000100:00100000:2.0:1776189609.660972:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f34aeb18 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664376512:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189609.660991:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189609.660991:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e9a35d2f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664376448:12345-0@lo:13:osp-pre-0-0.0 Request processed in 81us (432us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189609.661001:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6968 00000100:00100000:2.0:1776189609.661027:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f34aeb18 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664376512:12345-0@lo:13:osp-pre-1-0.0 Request processed in 54us (130us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189609.661035:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6969 00000100:00100000:2.0:1776189609.661079:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005ed146ab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664376512:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189609.661193:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c4b16b97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664376448:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189609.692301:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189609.948334:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189609.980365:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189610.204288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189610.460304:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:2.0:1776189610.684597:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189610.684604:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189610.684608:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189610.684616:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9752 00000100:00080000:2.0:1776189610.684620:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776189610.684625:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189610.684645:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005ee0b9e5 x1862459664376768/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189610.684658:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004ac5ba05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664376576:0@lo:400:kworker/u8:1.0 00000100:00080000:2.0:1776189610.684667:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189610.684670:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189610.684672:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189610.684676:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9752 00000100:00080000:2.0:1776189610.684678:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776189610.684681:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189610.684691:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000048c7c2e3 x1862459664376896/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189610.684699:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189610.684702:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189610.684703:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189610.684706:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9752 00000100:00080000:2.0:1776189610.684708:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776189610.684710:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189610.684719:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000023a5b7b3 x1862459664376960/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189610.684730:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189610.684767:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001a975e34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664377024:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189610.684795:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189610.684819:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000060a8344d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664377280:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189610.684839:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189610.684852:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1776189610.684857:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001a690fbd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664376832:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189610.684881:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189610.684897:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009015c790 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664377216:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189610.684916:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189610.684943:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664376832 00000100:00100000:1.0:1776189610.684970:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664377024 00000100:00100000:1.0:1776189610.685007:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664377280 00000100:00100000:1.0:1776189610.685024:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664377216 00000100:00100000:1.0:1776189610.685041:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8687 00000100:00100000:1.0:1776189610.685046:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008821baff pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664377024:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189610.685101:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008821baff pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664377024:12345-0@lo:400:kworker/u8:1.0 Request processed in 54us (306us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189610.685108:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8687 00000100:00100000:1.0:1776189610.685125:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8688 00000100:00100000:1.0:1776189610.685128:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e9138b66 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664377280:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189610.685153:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e9138b66 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664377280:12345-0@lo:400:kworker/u8:1.0 Request processed in 25us (315us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189610.685159:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8688 00000100:00100000:3.0:1776189610.685165:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10129 00000100:00100000:3.0:1776189610.685191:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000917ca813 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664376832:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189610.685205:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8689 00000100:00100000:1.0:1776189610.685210:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005355735b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664377216:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189610.685225:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000917ca813 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664376832:12345-0@lo:400:kworker/u8:1.0 Request processed in 53us (343us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189610.685232:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10129 00000100:00100000:1.0:1776189610.685261:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005355735b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664377216:12345-0@lo:400:kworker/u8:1.0 Request processed in 50us (344us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189610.685268:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8689 00000100:00100000:3.0:1776189610.685284:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001a690fbd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664376832:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189610.685297:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009015c790 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664377216:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189610.685301:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664376576 00000100:00100000:3.0:1776189610.685307:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001a975e34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664377024:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189610.685320:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000060a8344d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664377280:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189610.685331:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1543 00000100:00100000:1.0:1776189610.685336:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f07038bd pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664376576:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189610.685450:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f07038bd pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664376576:12345-0@lo:400:kworker/u8:1.0 Request processed in 113us (722us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189610.685456:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1543 00000100:00100000:0.0:1776189610.685826:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b39ea587 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664376640:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189610.685965:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189610.685991:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001fe1325f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664377088:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189610.686011:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189610.686032:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fc8a832b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664377344:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189610.686053:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189610.686070:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1776189610.686075:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e96e3d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664376704:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189610.686093:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189610.686105:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005bd91e44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664377152:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189610.686123:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189610.686144:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005ee0b9e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664376768:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189610.686162:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189610.686194:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000048c7c2e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664376896:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189610.686216:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189610.686249:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000023a5b7b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664376960:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189610.686270:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189610.686299:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664376960 00000100:00100000:0.0:1776189610.686323:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9166 00000100:00100000:0.0:1776189610.686327:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000a2844db pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664376960:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189610.686341:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189610.686641:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664376640 00000100:00100000:3.0:1776189610.686669:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664377152 00000100:00100000:3.0:1776189610.686684:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664376768 00000100:00100000:3.0:1776189610.686692:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664376896 00000100:00100000:3.0:1776189610.686702:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10130 00000100:00100000:3.0:1776189610.686705:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000badcd915 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664376640:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189610.686751:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000badcd915 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664376640:12345-0@lo:400:kworker/u8:1.0 Request processed in 43us (785us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189610.686758:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10130 00000100:00100000:3.0:1776189610.686775:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10131 00000100:00100000:0.0:1776189610.686779:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000a2844db pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664376960:12345-0@lo:38:kworker/u8:1.0 Request processed in 447us (505us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189610.686783:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c9d5f4ba pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664377152:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189610.686793:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9166 00000100:00100000:3.0:1776189610.686816:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c9d5f4ba pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664377152:12345-0@lo:400:kworker/u8:1.0 Request processed in 33us (691us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189610.686822:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10131 00000100:00100000:3.0:1776189610.686832:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57500 00000100:00100000:3.0:1776189610.686835:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d5486d87 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664376768:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189610.686844:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189610.686863:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d5486d87 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664376768:12345-0@lo:38:kworker/u8:1.0 Request processed in 27us (701us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189610.686868:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57500 00000100:00100000:3.0:1776189610.686877:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57501 00000100:00100000:3.0:1776189610.686880:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000270f1be6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664376896:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189610.686885:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189610.686902:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000270f1be6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664376896:12345-0@lo:38:kworker/u8:1.0 Request processed in 22us (687us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189610.686906:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57501 00000100:00100000:3.0:1776189610.686941:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005bd91e44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664377152:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189610.686965:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b39ea587 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664376640:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189610.686995:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664377344 00000100:00100000:2.0:1776189610.687006:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004ac5ba05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664376576:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189610.687018:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664376704 00000100:00080000:0.0:1776189610.687126:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189610.687131:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189610.687134:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005ee0b9e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664376768:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776189610.687143:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189610.687145:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189610.687147:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000048c7c2e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664376896:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776189610.687153:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189610.687156:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189610.687157:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000023a5b7b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664376960:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189610.687269:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664377088 00000100:00100000:1.0:1776189610.687304:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8690 00000100:00100000:1.0:1776189610.687309:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000062c76cbe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664377088:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189610.687505:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001fe1325f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664377088:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189610.687541:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000062c76cbe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664377088:12345-0@lo:400:kworker/u8:1.0 Request processed in 43us (1341us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189610.687550:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8690 00000100:00100000:3.0:1776189610.688217:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6970 00000100:00100000:3.0:1776189610.688224:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003f96cf0d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664377344:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776189610.688252:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189610.688297:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003f96cf0d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664377344:12345-0@lo:13:osp-pre-0-1.0 Request processed in 73us (2244us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189610.688303:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6970 00000100:00100000:3.0:1776189610.688318:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6971 00000100:00100000:3.0:1776189610.688322:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000af42fd42 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664376704:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189610.688333:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189610.688367:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000af42fd42 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664376704:12345-0@lo:13:osp-pre-1-1.0 Request processed in 44us (2273us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189610.688372:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6971 00000100:00100000:2.0:1776189610.693746:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e96e3d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664376704:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189610.693769:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fc8a832b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664377344:0@lo:13:osp-pre-0-1.0 00010000:02020000:0.0:1776189610.717333:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189610.972298:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189611.004592:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189611.231713:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189611.484291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189611.741735:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189611.996300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189612.028429:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189612.252280:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189612.508283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189612.764384:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189613.020299:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189613.052358:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189613.277081:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189613.532438:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189613.788316:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189614.044301:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189614.079544:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189614.300322:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189614.458061:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189614.458167:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189614.458313:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654962112 00000100:00100000:1.0:1776189614.458421:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1544 00000100:00100000:1.0:1776189614.458429:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005039a00d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654962112:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189614.458430:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654962240 00000100:00100000:3.0:1776189614.458463:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57502 00000100:00100000:3.0:1776189614.458469:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000822c622a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654962240:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:3.0:1776189614.458487:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189614.458516:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005039a00d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654962112:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 86us (463us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189614.458527:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000822c622a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654962240:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 54us (358us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189614.458528:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1544 00000100:00100000:3.0:1776189614.458538:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57502 00000100:00100000:2.0:1776189614.458627:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189614.459433:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654962176 00000100:00100000:3.0:1776189614.459471:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10132 00000100:00100000:3.0:1776189614.459475:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000739e313a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654962176:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189614.459526:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000739e313a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654962176:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 49us (899us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189614.459533:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10132 00010000:02020000:0.0:1776189614.557376:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189614.780579:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776189614.780604:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e8768bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664377408:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189614.780685:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189614.780784:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664377408 00000100:00100000:3.0:1776189614.780839:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6972 00000100:00100000:3.0:1776189614.780845:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007deebc61 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664377408:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189614.780876:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189614.780926:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007deebc61 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664377408:12345-0@lo:13:osp-pre-0-0.0 Request processed in 81us (241us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189614.780935:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6972 00000100:00100000:0.0:1776189614.781032:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e8768bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664377408:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189614.781369:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c90bb74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664377472:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189614.781408:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189614.781480:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664377472 00000100:00100000:1.0:1776189614.781518:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6973 00000100:00100000:1.0:1776189614.781522:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005bd91e44 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664377472:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189614.781542:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189614.781584:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005bd91e44 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664377472:12345-0@lo:13:osp-pre-1-0.0 Request processed in 61us (177us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189614.781590:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6973 00000100:00100000:0.0:1776189614.781658:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c90bb74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664377472:0@lo:13:osp-pre-1-0.0 00010000:02020000:0.0:1776189614.816533:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189615.068314:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189615.102901:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189615.324916:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189615.580294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189615.806807:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776189615.806828:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fc719d16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664377600:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189615.806895:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189615.806977:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664377600 00000100:00100000:3.0:1776189615.807024:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6974 00000100:00100000:3.0:1776189615.807030:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009015c790 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664377600:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776189615.807051:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189615.807090:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009015c790 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664377600:12345-0@lo:13:osp-pre-0-1.0 Request processed in 59us (195us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189615.807097:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6974 00000100:00100000:2.0:1776189615.807216:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fc719d16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664377600:0@lo:13:osp-pre-0-1.0 00000100:00080000:0.0:1776189615.807810:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189615.807814:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189615.807817:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189615.807822:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9757 00000100:00080000:0.0:1776189615.807824:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00080000:0.0:1776189615.807827:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189615.807842:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002e7c1306 x1862459664377792/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189615.807861:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189615.807863:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189615.807865:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189615.807867:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9757 00000100:00080000:0.0:1776189615.807869:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00080000:0.0:1776189615.807871:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189615.807877:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000fe79c55d x1862459664377920/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189615.807883:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189615.807886:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189615.807887:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189615.807889:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9757 00000100:00080000:0.0:1776189615.807891:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 6s 00000100:00080000:0.0:1776189615.807892:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189615.807899:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ef6c1a8d x1862459664377984/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189615.807982:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ef7b5ef7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664377664:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189615.808018:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189615.808034:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005f17ec69 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664378112:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189615.808060:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189615.808076:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776189615.808081:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d787fee7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664378048:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189615.808106:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189615.808146:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664378112 00000100:00100000:3.0:1776189615.808194:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664378048 00000100:00100000:3.0:1776189615.808342:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8691 00000100:00100000:3.0:1776189615.808385:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008dfea28a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664378112:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189615.808433:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008dfea28a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664378112:12345-0@lo:400:kworker/u8:0.0 Request processed in 84us (372us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189615.808440:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8691 00000100:00100000:3.0:1776189615.808453:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8692 00000100:00100000:3.0:1776189615.808457:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002eebf598 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664378048:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189615.808480:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002eebf598 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664378048:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (374us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189615.808486:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8692 00000100:00100000:3.0:1776189615.808532:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d787fee7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664378048:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189615.808542:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f17ec69 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664378112:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189615.808783:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000aef68a9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664377728:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189615.808813:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189615.808829:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000483ba61b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664378176:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189615.808848:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189615.808864:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:3.0:1776189615.808870:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000060a8344d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664377536:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189615.808891:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189615.808905:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000504398bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664377856:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189615.808927:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189615.808942:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e7453c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664378240:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189615.808951:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002e7c1306 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664377792:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189615.808965:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189615.808988:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189615.809007:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe79c55d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664377920:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189615.809075:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664378240 00000100:00100000:1.0:1776189615.809101:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8693 00000100:00100000:1.0:1776189615.809104:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003e8768bb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664378240:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189615.809150:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003e8768bb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664378240:12345-0@lo:400:kworker/u8:0.0 Request processed in 44us (187us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189615.809156:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8693 00000100:00100000:1.0:1776189615.809324:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e7453c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664378240:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189615.809348:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664377664 00000100:00100000:1.0:1776189615.809374:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1545 00000100:00100000:1.0:1776189615.809377:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000080a5522a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664377664:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189615.809414:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000080a5522a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664377664:12345-0@lo:400:kworker/u8:0.0 Request processed in 36us (1396us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189615.809420:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1545 00000100:00100000:0.0:1776189615.809578:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189615.810839:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664377728 00000100:00100000:3.0:1776189615.810890:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664378176 00000100:00100000:3.0:1776189615.810988:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664377856 00000100:00100000:3.0:1776189615.811002:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664377792 00000100:00100000:3.0:1776189615.811018:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10133 00000100:00100000:3.0:1776189615.811022:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000debe3528 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664377728:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189615.811086:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000debe3528 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664377728:12345-0@lo:400:kworker/u8:0.0 Request processed in 63us (2273us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189615.811093:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10133 00000100:00100000:3.0:1776189615.811114:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10134 00000100:00100000:3.0:1776189615.811118:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001ada75b0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664378176:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189615.811156:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001ada75b0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664378176:12345-0@lo:400:kworker/u8:0.0 Request processed in 37us (2308us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189615.811162:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10134 00000100:00100000:3.0:1776189615.811194:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10135 00000100:00100000:3.0:1776189615.811197:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fc719d16 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664377856:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189615.811231:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fc719d16 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664377856:12345-0@lo:400:kworker/u8:0.0 Request processed in 32us (2305us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189615.811237:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10135 00000100:00100000:3.0:1776189615.811249:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57503 00000100:00100000:3.0:1776189615.811653:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c229f2c4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664377792:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189615.811670:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189615.811701:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c229f2c4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664377792:12345-0@lo:38:kworker/u8:0.0 Request processed in 448us (2715us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189615.811708:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57503 00000100:00100000:3.0:1776189615.811770:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000504398bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664377856:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189615.811782:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000aef68a9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664377728:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189615.811792:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000483ba61b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664378176:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189615.811857:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ef7b5ef7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664377664:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189615.811896:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664377536 00000100:00100000:3.0:1776189615.811938:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6975 00000100:00100000:3.0:1776189615.811943:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007c231a0e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664377536:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189615.811960:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189615.812000:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007c231a0e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664377536:12345-0@lo:13:osp-pre-1-1.0 Request processed in 56us (3109us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189615.812006:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6975 00000100:00100000:3.0:1776189615.812042:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000060a8344d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664377536:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189615.816066:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ef6c1a8d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664377984:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189615.816144:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776189615.816280:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189615.816284:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189615.816289:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002e7c1306 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664377792:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189615.816394:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664377984 00000100:00100000:3.0:1776189615.816475:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9167 00000100:00100000:3.0:1776189615.816481:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000605c892b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664377984:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189615.816498:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189615.816556:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000605c892b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664377984:12345-0@lo:38:kworker/u8:0.0 Request processed in 74us (413us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189615.816563:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9167 00000100:00080000:0.0:1776189615.816868:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189615.816872:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189615.816875:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ef6c1a8d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664377984:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189615.823074:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664377920 00000100:00100000:2.0:1776189615.823159:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57504 00000100:00100000:2.0:1776189615.823166:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000021504d9d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664377920:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189615.823196:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189615.823426:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000021504d9d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664377920:12345-0@lo:38:kworker/u8:0.0 Request processed in 260us (13848us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189615.823434:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57504 00000100:00080000:0.0:1776189615.823745:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189615.823749:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189615.823754:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe79c55d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664377920:0@lo:38:kworker/u8:0.0 00010000:02020000:0.0:1776189615.836290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189616.092344:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189616.125269:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189616.348407:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189616.604985:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189616.860949:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189617.116509:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189617.148927:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189617.374509:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189617.629214:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189617.884294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189618.140295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189618.173774:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189618.397744:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189618.652272:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189618.908294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189619.164251:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189619.197425:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189619.420331:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189619.578737:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189619.578790:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189619.578858:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654962304 00000100:00100000:3.0:1776189619.578866:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654962368 00000100:00100000:1.0:1776189619.578921:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1546 00000100:00100000:3.0:1776189619.578927:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10136 00000100:00100000:1.0:1776189619.578931:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f99b7a85 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654962304:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1776189619.578934:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000060a8344d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654962368:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1776189619.579081:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000060a8344d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654962368:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 147us (353us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189619.579089:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10136 00000100:00100000:1.0:1776189619.579452:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f99b7a85 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654962304:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 521us (660us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189619.579467:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1546 00000100:00100000:3.0:1776189619.580829:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189619.581571:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654962432 00000100:00100000:1.0:1776189619.581732:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57505 00000100:00100000:1.0:1776189619.581738:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fc8a832b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654962432:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:1.0:1776189619.581758:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189619.581809:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fc8a832b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654962432:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 70us (986us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189619.581816:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57505 00010000:02020000:0.0:1776189619.676293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189619.900646:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776189619.900667:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e96e3d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664378304:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189619.900732:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189619.900861:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664378304 00000100:00100000:3.0:1776189619.900910:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6976 00000100:00100000:3.0:1776189619.900916:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e5ef59b0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664378304:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189619.900944:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189619.900992:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e5ef59b0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664378304:12345-0@lo:13:osp-pre-0-0.0 Request processed in 75us (261us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189619.900999:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6976 00000100:00100000:0.0:1776189619.901233:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e96e3d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664378304:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189619.901985:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a7f63c2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664378368:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189619.902025:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189619.902142:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664378368 00000100:00100000:2.0:1776189619.902202:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6977 00000100:00100000:2.0:1776189619.902207:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fe79c55d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664378368:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189619.902228:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189619.907115:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fe79c55d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664378368:12345-0@lo:13:osp-pre-1-0.0 Request processed in 4896us (5079us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189619.907134:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6977 00000100:00100000:0.0:1776189619.907501:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a7f63c2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664378368:0@lo:13:osp-pre-1-0.0 00010000:02020000:0.0:1776189619.933019:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189620.188835:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189620.221444:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189620.445169:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189620.700284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189620.924417:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c4b16b97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664378432:0@lo:13:osp-pre-0-1.0 00000100:00080000:0.0:1776189620.924478:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189620.924483:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189620.924487:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189620.924493:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9762 00000100:00100000:1.0:1776189620.924498:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776189620.924499:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00080000:0.0:1776189620.924506:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189620.924526:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ef7b5ef7 x1862459664378688/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189620.924548:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189620.924551:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189620.924552:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189620.924555:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9762 00000100:00080000:0.0:1776189620.924557:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:0.0:1776189620.924560:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189620.924567:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000aef68a9 x1862459664378816/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189620.924574:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189620.924576:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189620.924578:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189620.924580:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9762 00000100:00080000:0.0:1776189620.924582:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 11s 00000100:00080000:0.0:1776189620.924584:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189620.924594:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000504398bf x1862459664378880/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189620.924965:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000279e850a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664378496:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189620.924994:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189620.925010:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000483ba61b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664378752:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189620.925040:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189620.925058:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776189620.925062:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e7453c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664378944:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189620.925081:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189620.925107:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664378752 00000100:00100000:3.0:1776189620.925986:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10137 00000100:00100000:3.0:1776189620.925993:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000603f73df pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664378752:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189620.926101:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000603f73df pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664378752:12345-0@lo:400:kworker/u8:1.0 Request processed in 108us (1061us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189620.926109:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10137 00000100:00100000:3.0:1776189620.926151:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000483ba61b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664378752:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189620.926215:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ef6c1a8d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664378560:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189620.926245:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189620.926263:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776189620.926268:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002e7c1306 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664378624:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189620.926288:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189620.926312:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664378432 00000100:00100000:3.0:1776189620.926340:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664378496 00000100:00100000:3.0:1776189620.926356:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6978 00000100:00100000:3.0:1776189620.926360:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000746ef9bb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664378432:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776189620.926375:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189620.926849:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6979 00000100:00100000:2.0:1776189620.926856:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ff5d45b7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664378496:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189620.926870:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189620.926880:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664378560 00000100:00100000:2.0:1776189620.926920:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ff5d45b7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664378496:12345-0@lo:13:osp-pre-1-1.0 Request processed in 61us (1923us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189620.926921:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1547 00000100:00100000:1.0:1776189620.926928:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000afb92d61 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664378560:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189620.926932:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6979 00000100:00100000:3.0:1776189620.926957:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000746ef9bb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664378432:12345-0@lo:13:osp-pre-0-1.0 Request processed in 594us (2461us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189620.926965:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6978 00000100:00100000:1.0:1776189620.926970:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000afb92d61 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664378560:12345-0@lo:400:kworker/u8:1.0 Request processed in 42us (725us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189620.926976:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1547 00000100:00100000:2.0:1776189620.926981:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000279e850a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664378496:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189620.927001:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ef6c1a8d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664378560:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189620.927035:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c4b16b97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664378432:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189620.927037:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664378624 00000100:00100000:2.0:1776189620.927060:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664378944 00000100:00100000:3.0:1776189620.927063:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10138 00000100:00100000:3.0:1776189620.927068:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000611a3c66 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664378624:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189620.927071:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ef7b5ef7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664378688:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189620.927094:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8694 00000100:00100000:1.0:1776189620.927099:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189620.927102:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ac2fe2ac pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664378944:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189620.927105:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000611a3c66 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664378624:12345-0@lo:400:kworker/u8:1.0 Request processed in 34us (815us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189620.927114:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000aef68a9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664378816:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189620.927114:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10138 00000100:00100000:1.0:1776189620.927145:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189620.927153:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ac2fe2ac pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664378944:12345-0@lo:400:kworker/u8:1.0 Request processed in 52us (2070us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189620.927161:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8694 00000100:00100000:1.0:1776189620.927165:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000504398bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664378880:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189620.927208:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189620.927212:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e7453c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664378944:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189620.927253:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002e7c1306 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664378624:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189620.927271:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664378688 00000100:00100000:2.0:1776189620.927288:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664378816 00000100:00100000:2.0:1776189620.927302:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57506 00000100:00100000:2.0:1776189620.927305:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000029c2d1d5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664378688:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189620.927317:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189620.927346:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000029c2d1d5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664378688:12345-0@lo:38:kworker/u8:1.0 Request processed in 40us (251us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189620.927352:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57506 00000100:00100000:2.0:1776189620.927363:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57507 00000100:00100000:2.0:1776189620.927367:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000331ec920 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664378816:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189620.927374:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00080000:1.0:1776189620.927388:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189620.927393:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189620.927398:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ef7b5ef7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664378688:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189620.927402:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000331ec920 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664378816:12345-0@lo:38:kworker/u8:1.0 Request processed in 32us (256us total) trans 0 rc -19/-19 00000100:00080000:1.0:1776189620.927410:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189620.927414:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189620.927414:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57507 00000100:00100000:1.0:1776189620.927420:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000aef68a9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664378816:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189620.927446:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664378880 00000100:00100000:2.0:1776189620.927474:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9168 00000100:00100000:2.0:1776189620.927479:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009e8b9a01 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664378880:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189620.927487:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189620.927517:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009e8b9a01 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664378880:12345-0@lo:38:kworker/u8:1.0 Request processed in 37us (310us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189620.927524:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9168 00000100:00080000:1.0:1776189620.927623:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189620.927627:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189620.927630:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000504398bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664378880:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189620.927888:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d787fee7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664379072:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189620.927916:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189620.927933:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005f17ec69 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664379008:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189620.927957:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189620.928074:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664379072 00000100:00100000:3.0:1776189620.928107:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10139 00000100:00100000:3.0:1776189620.928111:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000504398bf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664379072:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189620.928146:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000504398bf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664379072:12345-0@lo:400:kworker/u8:1.0 Request processed in 34us (229us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189620.928152:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10139 00000100:00100000:3.0:1776189620.928194:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d787fee7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664379072:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189620.928382:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664379008 00000100:00100000:0.0:1776189620.928413:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8695 00000100:00100000:0.0:1776189620.928417:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000058fbcadb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664379008:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189620.928449:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000058fbcadb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664379008:12345-0@lo:400:kworker/u8:1.0 Request processed in 31us (491us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189620.928454:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8695 00000100:00100000:3.0:1776189620.928517:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f17ec69 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664379008:0@lo:400:kworker/u8:1.0 00010000:02020000:0.0:1776189620.957220:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189621.212296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189621.246565:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189621.468289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189621.724730:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189621.981143:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189622.237265:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189622.271416:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189622.493607:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189622.748460:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189623.004283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189623.260250:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189623.292742:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189623.516283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189623.773802:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189624.028313:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189624.284308:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189624.316624:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189624.542783:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189624.711064:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189624.711114:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189624.711270:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654962624 00000100:00100000:3.0:1776189624.711318:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57508 00000100:00100000:3.0:1776189624.711325:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000279e850a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654962624:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:3.0:1776189624.711344:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776189624.711423:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000279e850a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654962624:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 97us (308us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189624.711430:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57508 00000100:00100000:0.0:1776189624.713235:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189624.713332:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654962560 00000100:00100000:2.0:1776189624.713385:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10140 00000100:00100000:2.0:1776189624.713391:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000aef68a9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654962560:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776189624.713474:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000aef68a9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654962560:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 82us (245us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189624.713482:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10140 00000100:00100000:1.0:1776189624.714470:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654962496 00000100:00100000:1.0:1776189624.714522:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1548 00000100:00100000:1.0:1776189624.714528:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008f85e8f7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654962496:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776189624.714588:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008f85e8f7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654962496:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 60us (3532us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189624.714595:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1548 00010000:02020000:0.0:1776189624.796479:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189625.020503:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776189625.020526:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000081598856 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664379136:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189625.020598:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189625.020700:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664379136 00000100:00100000:1.0:1776189625.020750:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6980 00000100:00100000:1.0:1776189625.020755:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ef7b5ef7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664379136:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189625.020785:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189625.020826:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ef7b5ef7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664379136:12345-0@lo:13:osp-pre-1-0.0 Request processed in 70us (228us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189625.020833:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6980 00000100:00100000:0.0:1776189625.021294:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000081598856 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664379136:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189625.023433:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f1ff984b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664379200:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189625.023577:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189625.023753:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664379200 00000100:00100000:1.0:1776189625.023807:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6981 00000100:00100000:1.0:1776189625.023814:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002e7c1306 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664379200:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189625.023843:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189625.024076:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002e7c1306 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664379200:12345-0@lo:13:osp-pre-0-0.0 Request processed in 261us (501us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189625.024085:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6981 00000100:00100000:0.0:1776189625.024153:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f1ff984b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664379200:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189625.053752:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189625.308283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189625.340468:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189625.565055:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189625.820853:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189626.044677:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009257ba59 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664379264:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189626.044746:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189626.045152:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776189626.045162:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f1ff984b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664379328:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189626.045194:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664379264 00000100:00100000:3.0:1776189626.045209:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189626.045242:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664379328 00000100:00100000:1.0:1776189626.045260:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6982 00000100:00100000:1.0:1776189626.045265:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006bb8567a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664379264:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189626.045294:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189626.045345:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006bb8567a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664379264:12345-0@lo:13:osp-pre-1-1.0 Request processed in 79us (600us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189626.045355:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6982 00000100:00100000:1.0:1776189626.045370:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6983 00000100:00100000:1.0:1776189626.045373:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c4b16b97 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664379328:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189626.045383:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189626.045407:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c4b16b97 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664379328:12345-0@lo:13:osp-pre-0-1.0 Request processed in 33us (198us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189626.045413:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6983 00000100:00100000:3.0:1776189626.045816:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f1ff984b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664379328:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189626.045838:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009257ba59 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664379264:0@lo:13:osp-pre-1-1.0 00000100:00080000:1.0:1776189626.047369:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189626.047376:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189626.047379:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189626.047385:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9767 00000100:00080000:1.0:1776189626.047389:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:1.0:1776189626.047394:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189626.047411:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001bdbecc8 x1862459664379520/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189626.047447:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189626.047450:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189626.047452:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189626.047455:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9767 00000100:00080000:1.0:1776189626.047457:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00080000:1.0:1776189626.047460:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189626.047470:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d1846b23 x1862459664379648/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189626.047477:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189626.047482:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189626.047483:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189626.047486:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9767 00000100:00080000:1.0:1776189626.047488:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00080000:1.0:1776189626.047490:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189626.047500:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004ef5af0c x1862459664379712/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189626.047606:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3e4a548 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664379776:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189626.047667:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189626.047706:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776189626.047722:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000060733ebe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664379584:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189626.047747:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189626.047773:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f1df203a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664379456:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189626.047794:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189626.047809:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a7f63c2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664379904:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189626.047829:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189626.047846:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1776189626.047852:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b705f0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664379392:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189626.047875:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001bdbecc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664379520:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189626.047883:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189626.047902:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000081598856 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664379840:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189626.047933:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189626.047944:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189626.047956:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d1846b23 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664379648:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189626.047984:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189626.047985:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664379584 00000100:00100000:3.0:1776189626.048000:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004ef5af0c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664379712:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189626.048027:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189626.048032:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664379456 00000100:00100000:0.0:1776189626.048050:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664379904 00000100:00100000:3.0:1776189626.048061:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664379520 00000100:00100000:0.0:1776189626.048065:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664379648 00000100:00100000:0.0:1776189626.048087:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10141 00000100:00100000:0.0:1776189626.048093:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ef6c1a8d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664379584:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189626.048144:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ef6c1a8d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664379584:12345-0@lo:400:kworker/u8:0.0 Request processed in 50us (396us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189626.048210:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664379776 00000100:00100000:1.0:1776189626.048235:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664379840 00000100:00100000:1.0:1776189626.048252:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8696 00000100:00100000:1.0:1776189626.048255:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000e7453c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664379776:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189626.048289:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000e7453c0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664379776:12345-0@lo:400:kworker/u8:0.0 Request processed in 33us (623us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189626.048295:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8696 00000100:00100000:1.0:1776189626.048312:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8697 00000100:00100000:1.0:1776189626.048315:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d787fee7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664379840:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189626.048346:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d787fee7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664379840:12345-0@lo:400:kworker/u8:0.0 Request processed in 30us (404us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189626.048352:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8697 00000100:00100000:3.0:1776189626.053821:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10142 00000100:00100000:3.0:1776189626.053843:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000483ba61b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664379456:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189626.053955:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000483ba61b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664379456:12345-0@lo:400:kworker/u8:0.0 Request processed in 122us (6160us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189626.053963:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10142 00000100:00100000:3.0:1776189626.053995:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10143 00000100:00100000:3.0:1776189626.053999:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f17ec69 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664379904:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189626.054029:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f17ec69 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664379904:12345-0@lo:400:kworker/u8:0.0 Request processed in 29us (6200us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189626.054035:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10143 00000100:00100000:3.0:1776189626.054047:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57509 00000100:00100000:3.0:1776189626.054051:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f1ff984b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664379520:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189626.054066:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189626.054102:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f1ff984b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664379520:12345-0@lo:38:kworker/u8:0.0 Request processed in 50us (6170us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189626.054108:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57509 00000100:00100000:3.0:1776189626.054119:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57510 00000100:00100000:3.0:1776189626.054122:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006e96e3d4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664379648:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189626.054129:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189626.054148:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006e96e3d4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664379648:12345-0@lo:38:kworker/u8:0.0 Request processed in 25us (6165us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189626.054153:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57510 00000100:00080000:3.0:1776189626.054213:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189626.054217:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189626.054222:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001bdbecc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664379520:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776189626.054232:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189626.054234:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189626.054237:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d1846b23 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664379648:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189626.054296:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664379712 00000100:00100000:3.0:1776189626.054345:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9169 00000100:00100000:3.0:1776189626.054348:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001fe1325f pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664379712:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189626.054355:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189626.054385:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001fe1325f pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664379712:12345-0@lo:38:kworker/u8:0.0 Request processed in 35us (6359us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189626.054390:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9169 00000100:00080000:3.0:1776189626.054410:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189626.054413:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189626.054415:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004ef5af0c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664379712:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189626.054462:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664379392 00000100:00100000:3.0:1776189626.054490:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1549 00000100:00100000:3.0:1776189626.054493:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000034f05964 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664379392:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189626.056275:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000060733ebe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664379584:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189626.056294:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3e4a548 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664379776:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189626.057218:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000081598856 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664379840:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189626.057235:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f1df203a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664379456:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189626.057246:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a7f63c2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664379904:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189626.059682:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000034f05964 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664379392:12345-0@lo:400:kworker/u8:0.0 Request processed in 5180us (11794us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189626.059698:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1549 00000100:00100000:2.0:1776189626.059737:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b705f0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664379392:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189626.062659:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10141 00010000:02020000:0.0:1776189626.076832:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189626.332351:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189626.364728:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189626.588289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189626.844319:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189627.100662:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189627.358567:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189627.388725:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189627.612291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189627.879292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189628.132291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189628.388416:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189628.412666:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189628.644479:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189628.900722:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189629.161786:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189629.412502:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189629.436652:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189629.668287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189629.821282:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189629.821339:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189629.821438:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189629.821454:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654962688 00000100:00100000:3.0:1776189629.821513:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1550 00000100:00100000:3.0:1776189629.821519:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d6b85e7e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654962688:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1776189629.821582:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d6b85e7e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654962688:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 62us (242us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189629.821589:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1550 00000100:00100000:2.0:1776189629.821700:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654962752 00000100:00100000:2.0:1776189629.821751:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654962816 00000100:00100000:2.0:1776189629.821775:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10144 00000100:00100000:2.0:1776189629.821780:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007bf8226c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654962752:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776189629.821817:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007bf8226c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654962752:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 36us (543us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189629.821833:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10144 00000100:00100000:2.0:1776189629.821847:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57511 00000100:00100000:2.0:1776189629.821850:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000684e24e8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654962816:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:2.0:1776189629.821864:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189629.821889:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000684e24e8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654962816:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 38us (454us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189629.821895:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57511 00010000:02020000:2.0:1776189629.924302:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189630.140510:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000068939cd3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664379968:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189630.140586:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189630.140626:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664379968 00000100:00100000:3.0:1776189630.140633:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776189630.140640:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000023a5b7b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664380032:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189630.140666:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189630.143596:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664380032 00000100:00100000:1.0:1776189630.143814:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6984 00000100:00100000:1.0:1776189630.143876:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000048c7c2e3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664379968:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189630.143889:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6985 00000100:00100000:2.0:1776189630.143896:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005ee0b9e5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664380032:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189630.143922:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189630.143989:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005ee0b9e5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664380032:12345-0@lo:13:osp-pre-0-0.0 Request processed in 93us (3324us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189630.143996:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6985 00000100:00100000:3.0:1776189630.144063:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000023a5b7b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664380032:0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189630.144422:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189630.144556:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000048c7c2e3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664379968:12345-0@lo:13:osp-pre-1-0.0 Request processed in 722us (3971us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189630.144563:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6984 00000100:00100000:3.0:1776189630.158056:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000068939cd3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664379968:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189630.180348:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189630.437410:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189630.461904:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189630.692293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189630.948314:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:3.0:1776189631.164555:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189631.164563:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189631.164566:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189631.164575:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9773 00000100:00080000:3.0:1776189631.164579:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189631.164584:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189631.164602:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b39ea587 x1862459664380224/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189631.164632:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189631.164636:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189631.164637:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189631.164640:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9773 00000100:00080000:3.0:1776189631.164642:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189631.164645:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189631.164655:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001a690fbd x1862459664380352/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189631.164662:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189631.164666:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189631.164667:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189631.164670:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9773 00000100:00080000:3.0:1776189631.164672:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189631.164674:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189631.164686:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000075d6a9fc x1862459664380416/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189631.164722:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b39ea587 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664380224:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189631.164729:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001a975e34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664380288:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189631.164826:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004ac5ba05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664380160:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189631.164966:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000023a5b7b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664380096:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189631.167502:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189631.167545:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001a690fbd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664380352:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189631.167720:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189631.167747:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000075d6a9fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664380416:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189631.167759:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189631.167882:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189631.167888:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace99b9b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664380608:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189631.168042:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189631.168060:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c6f4c7a3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664380864:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189631.168084:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189631.168147:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664380864 00000100:00100000:1.0:1776189631.168204:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6986 00000100:00100000:1.0:1776189631.168210:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000f61bb34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664380864:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189631.168231:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189631.168574:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c90bb74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664380480:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189631.168603:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189631.168620:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000023fe30ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664380736:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189631.168638:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189631.168666:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664380224 00000100:00100000:2.0:1776189631.168689:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664380352 00000100:00100000:2.0:1776189631.168708:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664380160 00000100:00100000:1.0:1776189631.168728:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000f61bb34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664380864:12345-0@lo:13:osp-pre-1-1.0 Request processed in 515us (641us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189631.168739:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10145 00000100:00100000:1.0:1776189631.168744:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6986 00000100:00100000:2.0:1776189631.168747:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e0967c51 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664380160:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189631.168766:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664380480 00000100:00100000:1.0:1776189631.168791:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6987 00000100:00100000:1.0:1776189631.168805:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009fddac3a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664380480:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189631.168817:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189631.169129:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009fddac3a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664380480:12345-0@lo:13:osp-pre-0-1.0 Request processed in 332us (525us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189631.169136:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6987 00000100:00100000:3.0:1776189631.169258:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189631.169274:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000099b8eb0d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664380672:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189631.169299:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189631.169335:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664380288 00000100:00100000:3.0:1776189631.169352:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664380672 00000100:00100000:3.0:1776189631.169362:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10146 00000100:00100000:3.0:1776189631.169366:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f9e630d1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664380288:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189631.169401:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f9e630d1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664380288:12345-0@lo:400:kworker/u8:1.0 Request processed in 33us (144us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189631.169407:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10146 00000100:00100000:3.0:1776189631.169419:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10147 00000100:00100000:3.0:1776189631.169422:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a3202503 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664380672:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189631.169448:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a3202503 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664380672:12345-0@lo:400:kworker/u8:1.0 Request processed in 25us (149us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189631.169453:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10147 00000100:00100000:3.0:1776189631.169463:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57512 00000100:00100000:3.0:1776189631.169466:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005ed146ab pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664380224:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189631.169480:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189631.169510:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005ed146ab pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664380224:12345-0@lo:38:kworker/u8:1.0 Request processed in 43us (2016us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189631.169516:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57512 00000100:00100000:3.0:1776189631.169529:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57513 00000100:00100000:3.0:1776189631.169532:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000054387670 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664380352:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189631.169543:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189631.169558:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c90bb74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664380480:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189631.169572:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000054387670 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664380352:12345-0@lo:38:kworker/u8:1.0 Request processed in 36us (1991us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189631.169579:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57513 00000100:00100000:3.0:1776189631.169606:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001a975e34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664380288:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189631.169614:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000099b8eb0d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664380672:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189631.169734:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664380608 00000100:00100000:3.0:1776189631.169758:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664380736 00000100:00100000:1.0:1776189631.169818:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c6f4c7a3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664380864:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189631.170514:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8698 00000100:00100000:3.0:1776189631.170520:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000071c1b92f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664380608:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189631.170565:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000071c1b92f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664380608:12345-0@lo:400:kworker/u8:1.0 Request processed in 45us (2524us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189631.170571:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8698 00000100:00100000:3.0:1776189631.170583:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8699 00000100:00100000:3.0:1776189631.170586:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009257ba59 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664380736:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189631.170616:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009257ba59 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664380736:12345-0@lo:400:kworker/u8:1.0 Request processed in 29us (1977us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189631.170622:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8699 00000100:00100000:3.0:1776189631.170642:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664380416 00000100:00100000:3.0:1776189631.170661:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9170 00000100:00100000:3.0:1776189631.170665:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000545eaa39 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664380416:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189631.170674:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189631.170702:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000545eaa39 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664380416:12345-0@lo:38:kworker/u8:1.0 Request processed in 36us (2821us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189631.170708:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9170 00000100:00080000:3.0:1776189631.170848:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189631.170853:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189631.170857:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b39ea587 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664380224:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776189631.170867:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189631.170870:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189631.170872:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001a690fbd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664380352:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776189631.170880:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189631.170882:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189631.170884:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000075d6a9fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664380416:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189631.171255:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189631.171277:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4a5ce06 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664380544:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189631.171307:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189631.171334:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f8f9cfc6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664380800:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189631.171351:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000023fe30ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664380736:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189631.171358:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189631.171397:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664380544 00000100:00100000:1.0:1776189631.171402:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace99b9b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664380608:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189631.171426:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664380800 00000100:00100000:0.0:1776189631.171449:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8700 00000100:00100000:0.0:1776189631.171456:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022f2aa27 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664380544:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189631.171457:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8701 00000100:00100000:1.0:1776189631.171466:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000067c9d69b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664380800:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189631.171502:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022f2aa27 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664380544:12345-0@lo:400:kworker/u8:1.0 Request processed in 46us (193us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189631.171509:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8700 00000100:00100000:1.0:1776189631.171514:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000067c9d69b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664380800:12345-0@lo:400:kworker/u8:1.0 Request processed in 47us (154us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189631.171522:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8701 00000100:00100000:0.0:1776189631.171573:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4a5ce06 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664380544:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189631.171583:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f8f9cfc6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664380800:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189631.171787:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664380096 00000100:00100000:3.0:1776189631.171828:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1551 00000100:00100000:3.0:1776189631.171832:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004c446be7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664380096:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189631.171882:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004c446be7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664380096:12345-0@lo:400:kworker/u8:1.0 Request processed in 49us (628us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189631.171888:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1551 00000100:00100000:1.0:1776189631.172252:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000023a5b7b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664380096:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189631.172440:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e0967c51 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664380160:12345-0@lo:400:kworker/u8:1.0 Request processed in 3693us (4682us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189631.172449:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10145 00000100:00100000:1.0:1776189631.172520:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004ac5ba05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664380160:0@lo:400:kworker/u8:1.0 00010000:02020000:2.0:1776189631.204456:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189631.460450:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189631.484881:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189631.718967:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189631.972264:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189632.230398:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189632.484372:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189632.511761:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189632.740497:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189632.997700:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189633.254829:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189633.508865:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189633.532661:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189633.764291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189634.023297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189634.276285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189634.532327:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189634.556946:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189634.788382:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189634.939098:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189634.939806:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654963008 00000100:00100000:2.0:1776189634.939860:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57514 00000100:00100000:2.0:1776189634.939866:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000526ed979 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654963008:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776189634.939886:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189634.939953:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000526ed979 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654963008:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 86us (862us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189634.939961:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57514 00000100:00100000:0.0:1776189634.942530:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189634.942572:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189634.942654:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654962880 00000100:00100000:2.0:1776189634.942703:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1552 00000100:00100000:2.0:1776189634.942708:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fe4b4330 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654962880:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189634.942783:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fe4b4330 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654962880:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 74us (260us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189634.942790:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1552 00000100:00100000:3.0:1776189634.943674:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654962944 00000100:00100000:3.0:1776189634.943764:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10148 00000100:00100000:3.0:1776189634.943768:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d92ffa01 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654962944:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189634.943822:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d92ffa01 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654962944:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 53us (1250us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189634.943829:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10148 00010000:02020000:0.0:1776189635.044290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189635.260786:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776189635.270195:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009820cd1e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664380992:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189635.270483:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189635.270578:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003f67f1c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664380928:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189635.270614:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189635.282008:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664380992 00000100:00100000:3.0:1776189635.282086:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664380928 00000100:00100000:3.0:1776189635.282120:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6988 00000100:00100000:3.0:1776189635.282126:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c6f4c7a3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664380992:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776189635.282158:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189635.282350:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6989 00000100:00100000:0.0:1776189635.282357:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000412a95e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664380928:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776189635.282373:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189635.282431:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c6f4c7a3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664380992:12345-0@lo:13:osp-pre-1-0.0 Request processed in 304us (11948us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189635.282438:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6988 00000100:00100000:0.0:1776189635.283504:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000412a95e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664380928:12345-0@lo:13:osp-pre-0-0.0 Request processed in 1142us (12894us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189635.283514:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6989 00000100:00100000:2.0:1776189635.283660:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009820cd1e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664380992:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189635.283692:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003f67f1c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664380928:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189635.300776:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189635.556455:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189635.580509:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189635.812291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189636.068302:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189636.284438:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c90bb74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664381056:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189636.284507:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189636.284608:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664381056 00000100:00100000:1.0:1776189636.284672:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6990 00000100:00100000:1.0:1776189636.284679:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d02fb24b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664381056:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189636.284712:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00080000:3.0:1776189636.284747:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189636.284760:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189636.284765:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1776189636.284766:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d02fb24b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664381056:12345-0@lo:13:osp-pre-0-1.0 Request processed in 86us (259us total) trans 0 rc 0/0 00000100:00080000:3.0:1776189636.284772:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9778 00000100:00080000:3.0:1776189636.284776:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1776189636.284779:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6990 00000100:00080000:3.0:1776189636.284783:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189636.284807:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003f67f1c5 x1862459664381248/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189636.284820:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000026520930 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664381120:0@lo:400:kworker/u8:1.0 00000100:00080000:3.0:1776189636.284834:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189636.284839:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189636.284840:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189636.284844:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9778 00000100:00080000:3.0:1776189636.284846:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189636.284850:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189636.284855:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189636.284862:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000075d6a9fc x1862459664381376/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189636.284871:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189636.284874:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189636.284876:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189636.284880:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9778 00000100:00080000:3.0:1776189636.284883:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189636.284886:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189636.284890:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664381120 00000100:00100000:3.0:1776189636.284908:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ace99b9b x1862459664381440/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189636.284919:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1553 00000100:00100000:1.0:1776189636.284926:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000459e616f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664381120:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189636.284962:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003f67f1c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664381248:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189636.284983:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000459e616f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664381120:12345-0@lo:400:kworker/u8:1.0 Request processed in 56us (128us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189636.284991:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1553 00000100:00100000:3.0:1776189636.285000:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189636.285020:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000075d6a9fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664381376:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189636.285035:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000090d63ac9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664381184:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189636.285044:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189636.285062:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace99b9b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664381440:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189636.285068:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189636.285359:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000023fe30ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664381504:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189636.285407:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004ac5ba05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664381632:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189636.285495:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664381248 00000100:00100000:2.0:1776189636.285516:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664381376 00000100:00100000:2.0:1776189636.285530:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664381184 00000100:00100000:2.0:1776189636.285553:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10149 00000100:00100000:3.0:1776189636.285557:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189636.285561:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005b705f0f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664381184:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189636.285598:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57515 00000100:00100000:3.0:1776189636.285603:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001a690fbd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664381248:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189636.285607:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005b705f0f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664381184:12345-0@lo:400:kworker/u8:1.0 Request processed in 44us (538us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189636.285615:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10149 00010000:02000000:3.0:1776189636.285620:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189636.285630:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57516 00000100:00100000:2.0:1776189636.285633:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b39ea587 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664381376:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189636.285737:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001a690fbd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664381248:12345-0@lo:38:kworker/u8:1.0 Request processed in 133us (738us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189636.285884:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57515 00000100:00100000:3.0:1776189636.285910:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664381440 00000100:00100000:3.0:1776189636.286332:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9171 00000100:00100000:3.0:1776189636.286336:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000acc55d97 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664381440:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189636.286346:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189636.286382:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000acc55d97 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664381440:12345-0@lo:38:kworker/u8:1.0 Request processed in 44us (829us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189636.286388:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9171 00010000:02000000:2.0:1776189636.286392:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189636.286440:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b39ea587 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664381376:12345-0@lo:38:kworker/u8:1.0 Request processed in 804us (1396us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189636.286448:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57516 00000100:00100000:2.0:1776189636.286489:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4a5ce06 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664381312:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189636.286521:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189636.286542:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000023a5b7b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664381696:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189636.286572:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1776189636.286595:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189636.286599:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189636.286603:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003f67f1c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664381248:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189636.286613:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664381312 00000100:00080000:3.0:1776189636.286614:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189636.286617:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189636.286619:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000075d6a9fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664381376:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776189636.286625:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189636.286626:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189636.286629:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace99b9b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664381440:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189636.286728:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664381696 00000100:00100000:2.0:1776189636.286893:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10150 00000100:00100000:2.0:1776189636.286900:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f09d90e0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664381312:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189636.286940:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f09d90e0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664381312:12345-0@lo:400:kworker/u8:1.0 Request processed in 40us (419us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189636.286950:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10150 00000100:00100000:2.0:1776189636.286991:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4a5ce06 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664381312:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189636.287022:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000099b8eb0d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664381568:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189636.287045:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189636.287069:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000026520930 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664381120:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189636.287241:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776189636.287250:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e0bab4ab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664381760:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189636.287277:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189636.287312:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664381568 00000100:00100000:2.0:1776189636.287318:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8702 00000100:00100000:2.0:1776189636.287324:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000008468a4e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664381696:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189636.287340:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8703 00000100:00100000:3.0:1776189636.287345:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000748b663e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664381568:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189636.287362:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000008468a4e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664381696:12345-0@lo:400:kworker/u8:1.0 Request processed in 37us (789us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189636.287369:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8702 00000100:00100000:3.0:1776189636.287391:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000748b663e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664381568:12345-0@lo:400:kworker/u8:1.0 Request processed in 44us (344us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189636.287399:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8703 00000100:00100000:2.0:1776189636.287403:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000023a5b7b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664381696:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189636.287444:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000099b8eb0d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664381568:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189636.287472:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664381760 00000100:00100000:3.0:1776189636.287497:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6991 00000100:00100000:3.0:1776189636.287501:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000033264082 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664381760:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189636.287515:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189636.287563:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000033264082 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664381760:12345-0@lo:13:osp-pre-1-1.0 Request processed in 61us (285us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189636.287569:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6991 00000100:00100000:2.0:1776189636.287715:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e0bab4ab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664381760:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189636.288605:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189636.292560:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c90bb74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664381056:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189636.293569:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664381504 00000100:00100000:3.0:1776189636.293758:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8704 00000100:00100000:3.0:1776189636.293765:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009820cd1e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664381504:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189636.293851:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009820cd1e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664381504:12345-0@lo:400:kworker/u8:1.0 Request processed in 86us (5249us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189636.293858:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8704 00000100:00100000:1.0:1776189636.294851:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189636.294958:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000090d63ac9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664381184:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189636.295110:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664381632 00000100:00100000:2.0:1776189636.295166:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10151 00000100:00100000:2.0:1776189636.295170:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007a7f63c2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664381632:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189636.295279:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007a7f63c2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664381632:12345-0@lo:400:kworker/u8:1.0 Request processed in 107us (431us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189636.295286:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10151 00000100:00100000:1.0:1776189636.295600:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004ac5ba05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664381632:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189636.296955:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000023fe30ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664381504:0@lo:400:kworker/u8:1.0 00010000:02020000:2.0:1776189636.324938:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189636.580822:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189636.604460:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189636.837125:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189637.092400:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189637.348383:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189637.604317:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189637.628653:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189637.861321:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189638.116443:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189638.372314:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189638.628326:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189638.652998:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189638.884289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189639.140287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189639.396344:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189639.652255:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189639.676453:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189639.908501:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189640.066158:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189640.066228:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189640.066275:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189640.066324:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654963072 00000100:00100000:3.0:1776189640.066381:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1554 00000100:00100000:3.0:1776189640.066389:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000de62cc6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654963072:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776189640.066392:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654963200 00000100:00100000:1.0:1776189640.066423:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57517 00000100:00100000:1.0:1776189640.066429:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000074e02040 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654963200:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:1.0:1776189640.066445:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776189640.066461:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000de62cc6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654963072:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 72us (233us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189640.066469:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1554 00000100:00100000:1.0:1776189640.066477:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000074e02040 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654963200:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 47us (202us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189640.066484:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57517 00000100:00100000:2.0:1776189640.069262:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654963136 00000100:00100000:2.0:1776189640.069423:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10152 00000100:00100000:2.0:1776189640.069431:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000068939cd3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654963136:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776189640.070473:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000068939cd3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654963136:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 1036us (4317us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189640.070485:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10152 00010000:02020000:2.0:1776189640.164296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189640.380612:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c094ed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664381888:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189640.380612:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776189640.380631:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001fe1325f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664381824:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189640.380977:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189640.381157:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664381888 00000100:00100000:0.0:1776189640.381263:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6992 00000100:00100000:0.0:1776189640.381270:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d72c99d5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664381888:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189640.381299:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189640.381354:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d72c99d5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664381888:12345-0@lo:13:osp-pre-1-0.0 Request processed in 84us (377us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189640.381362:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6992 00000100:00100000:1.0:1776189640.381440:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c094ed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664381888:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189640.389669:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189640.391258:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664381824 00000100:00100000:0.0:1776189640.391349:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6993 00000100:00100000:0.0:1776189640.391356:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001a975e34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664381824:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776189640.391407:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189640.391701:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001a975e34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664381824:12345-0@lo:13:osp-pre-0-0.0 Request processed in 344us (2039us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189640.391708:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6993 00000100:00100000:3.0:1776189640.394290:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001fe1325f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664381824:0@lo:13:osp-pre-0-0.0 00010000:02020000:2.0:1776189640.421885:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189640.677747:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189640.700833:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189640.934640:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189641.188329:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189641.404679:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001fe1325f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664381952:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189641.404769:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189641.404824:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba6916cf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664382016:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189641.404852:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189641.404886:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664381952 00000100:00100000:0.0:1776189641.404927:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1555 00000100:00100000:0.0:1776189641.404933:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000082b19b4e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664381952:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189641.404978:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000082b19b4e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664381952:12345-0@lo:400:kworker/u8:0.0 Request processed in 44us (210us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189641.404985:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1555 00000100:00100000:0.0:1776189641.405019:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664382016 00000100:00100000:0.0:1776189641.405041:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6994 00000100:00100000:0.0:1776189641.405045:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f8f9cfc6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664382016:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189641.405059:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189641.405091:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f8f9cfc6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664382016:12345-0@lo:13:osp-pre-1-1.0 Request processed in 45us (239us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189641.405097:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6994 00000100:00100000:0.0:1776189641.405135:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba6916cf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664382016:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189641.405162:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001fe1325f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664381952:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189641.410450:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001fe1325f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664382144:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189641.410518:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1776189641.412852:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189641.412864:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189641.412867:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189641.412875:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9783 00000100:00080000:3.0:1776189641.412878:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776189641.412885:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189641.412923:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004ac5ba05 x1862459664382208/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189641.412953:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189641.412957:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189641.412959:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189641.412962:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9783 00000100:00080000:3.0:1776189641.412963:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776189641.412967:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189641.412980:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000013c0eaff x1862459664382336/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189641.412989:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189641.412992:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189641.412993:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189641.412996:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9783 00000100:00080000:3.0:1776189641.412997:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776189641.413000:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189641.413011:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007ec15c7b x1862459664382400/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189641.413124:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776189641.413134:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000023a5b7b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664382272:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189641.413200:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189641.413260:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664382144 00000100:00100000:2.0:1776189641.413304:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664382272 00000100:00100000:2.0:1776189641.413324:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10153 00000100:00100000:2.0:1776189641.413329:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009b827fed pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664382144:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189641.413388:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009b827fed pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664382144:12345-0@lo:400:kworker/u8:0.0 Request processed in 58us (2871us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189641.413395:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10153 00000100:00100000:2.0:1776189641.413410:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10154 00000100:00100000:2.0:1776189641.413413:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009ea5366f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664382272:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189641.413435:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009ea5366f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664382272:12345-0@lo:400:kworker/u8:0.0 Request processed in 21us (236us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189641.413440:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10154 00000100:00100000:2.0:1776189641.413474:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000023a5b7b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664382272:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189641.413486:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001fe1325f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664382144:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189641.413613:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a38efcb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664382464:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189641.413655:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189641.413680:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776189641.413686:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c094ed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664382080:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189641.413715:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189641.413738:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004ac5ba05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664382208:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189641.413759:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189641.413770:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000013c0eaff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664382336:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189641.413789:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189641.413800:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ec15c7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664382400:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189641.413864:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189641.413893:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664382208 00000100:00100000:2.0:1776189641.413914:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664382336 00000100:00100000:2.0:1776189641.413926:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57518 00000100:00100000:2.0:1776189641.413929:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000023a5b7b3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664382208:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189641.413944:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189641.413976:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000023a5b7b3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664382208:12345-0@lo:38:kworker/u8:0.0 Request processed in 45us (216us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189641.413981:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57518 00000100:00100000:2.0:1776189641.413991:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57519 00000100:00100000:2.0:1776189641.413994:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e0bab4ab pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664382336:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189641.414000:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189641.414018:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e0bab4ab pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664382336:12345-0@lo:38:kworker/u8:0.0 Request processed in 23us (229us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189641.414026:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57519 00000100:00100000:2.0:1776189641.414043:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664382080 00000100:00100000:2.0:1776189641.414064:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6995 00000100:00100000:2.0:1776189641.414068:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001fe1325f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664382080:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189641.414083:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189641.414116:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001fe1325f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664382080:12345-0@lo:13:osp-pre-0-1.0 Request processed in 47us (402us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189641.414122:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6995 00000100:00080000:2.0:1776189641.414146:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189641.414149:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189641.414152:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004ac5ba05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664382208:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776189641.414158:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189641.414160:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189641.414162:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000013c0eaff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664382336:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189641.414218:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c094ed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664382080:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189641.414254:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664382400 00000100:00100000:2.0:1776189641.414272:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9172 00000100:00100000:2.0:1776189641.414275:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000932d0506 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664382400:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189641.414284:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189641.414312:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000932d0506 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664382400:12345-0@lo:38:kworker/u8:0.0 Request processed in 36us (449us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189641.414318:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9172 00000100:00080000:2.0:1776189641.414340:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189641.414343:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189641.414347:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ec15c7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664382400:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189641.414388:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664382464 00000100:00100000:2.0:1776189641.414408:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8705 00000100:00100000:2.0:1776189641.414411:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba6916cf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664382464:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189641.414446:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba6916cf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664382464:12345-0@lo:400:kworker/u8:0.0 Request processed in 33us (791us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189641.414451:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8705 00000100:00100000:2.0:1776189641.414476:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a38efcb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664382464:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189641.419308:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f2a9137 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664382528:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189641.419384:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189641.419469:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664382528 00000100:00100000:2.0:1776189641.419515:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8706 00000100:00100000:2.0:1776189641.419521:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000041c094ed pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664382528:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189641.419650:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776189641.419662:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000036a45663 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664382592:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189641.419693:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189641.419722:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664382592 00000100:00100000:3.0:1776189641.419751:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10155 00000100:00100000:3.0:1776189641.419754:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000960dca04 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664382592:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189641.419788:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000960dca04 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664382592:12345-0@lo:400:kworker/u8:0.0 Request processed in 32us (95us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189641.419795:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10155 00000100:00100000:3.0:1776189641.419837:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000036a45663 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664382592:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189641.424369:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000041c094ed pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664382528:12345-0@lo:400:kworker/u8:0.0 Request processed in 4833us (4971us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189641.424385:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8706 00000100:00100000:0.0:1776189641.425787:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f2a9137 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664382528:0@lo:400:kworker/u8:0.0 00010000:02020000:2.0:1776189641.444299:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189641.700455:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189641.724648:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189641.956312:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189642.212798:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189642.468288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189642.725348:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189642.748609:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189642.980288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189643.237990:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189643.494271:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189643.748931:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189643.772489:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189644.004272:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189644.260293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189644.517447:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189644.774715:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189644.799859:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189645.028314:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189645.180373:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189645.182559:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654963392 00000100:00100000:1.0:1776189645.182625:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57520 00000100:00100000:1.0:1776189645.182632:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c0d0294 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654963392:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776189645.182653:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189645.182734:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c0d0294 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654963392:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 102us (2368us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189645.182742:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57520 00000100:00100000:0.0:1776189645.190589:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189645.190714:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189645.194164:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654963328 00000100:00100000:2.0:1776189645.194277:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10156 00000100:00100000:2.0:1776189645.194284:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e5117e34 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654963328:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189645.194370:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e5117e34 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654963328:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 85us (3656us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189645.194379:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10156 00000100:00100000:3.0:1776189645.195117:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654963264 00000100:00100000:3.0:1776189645.195160:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1556 00000100:00100000:3.0:1776189645.195165:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000055012753 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654963264:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189645.195375:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000055012753 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654963264:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 209us (4794us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189645.195474:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1556 00010000:02020000:2.0:1776189645.284319:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189645.500530:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776189645.500553:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ab3412dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664382656:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189645.500617:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189645.500837:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776189645.500846:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f1df203a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664382720:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189645.500872:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189645.500998:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664382656 00000100:00100000:1.0:1776189645.501043:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664382720 00000100:00100000:1.0:1776189645.501068:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6996 00000100:00100000:1.0:1776189645.501073:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000088c2a62e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664382656:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189645.501103:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189645.501143:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000088c2a62e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664382656:12345-0@lo:13:osp-pre-1-0.0 Request processed in 69us (526us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189645.501150:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6996 00000100:00100000:1.0:1776189645.501167:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6997 00000100:00100000:1.0:1776189645.501188:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d45cc7cd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664382720:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189645.501203:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189645.501231:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d45cc7cd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664382720:12345-0@lo:13:osp-pre-0-0.0 Request processed in 59us (359us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189645.501237:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6997 00000100:00100000:2.0:1776189645.501410:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f1df203a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664382720:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189645.501439:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ab3412dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664382656:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189645.540516:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189645.796438:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189645.820515:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189646.054005:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189646.309565:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:2.0:1776189646.526305:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189646.526311:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189646.526314:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189646.526321:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9788 00000100:00080000:2.0:1776189646.526325:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00080000:2.0:1776189646.526329:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189646.526346:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cf6b5d65 x1862459664382912/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189646.526367:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189646.526370:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189646.526371:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189646.526375:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9788 00000100:00080000:2.0:1776189646.526377:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00080000:2.0:1776189646.526380:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189646.526390:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007ec15c7b x1862459664383040/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189646.526397:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189646.526400:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189646.526401:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189646.526404:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9788 00000100:00080000:2.0:1776189646.526405:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 6s 00000100:00080000:2.0:1776189646.526409:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189646.526419:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000013c0eaff x1862459664383104/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189646.526483:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cf6b5d65 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664382912:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189646.526545:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189646.526569:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ec15c7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664383040:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189646.526603:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189646.526618:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000013c0eaff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664383104:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189646.526641:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189646.526680:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664383104 00000100:00100000:2.0:1776189646.526710:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9173 00000100:00100000:2.0:1776189646.526716:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001546cd51 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664383104:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189646.526731:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189646.526766:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001546cd51 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664383104:12345-0@lo:38:kworker/u8:1.0 Request processed in 50us (124us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189646.526773:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9173 00000100:00100000:2.0:1776189646.526794:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dd5e571c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664382848:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189646.526814:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189646.526825:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000026520930 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664383296:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189646.526840:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189646.526853:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1776189646.526857:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004dd53b72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664382976:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189646.527008:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664382912 00000100:00100000:3.0:1776189646.527030:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664383040 00000100:00100000:3.0:1776189646.527043:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664382848 00000100:00100000:3.0:1776189646.527060:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664383296 00000100:00100000:3.0:1776189646.527071:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10157 00000100:00100000:3.0:1776189646.527075:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000a2844db pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664382848:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189646.527114:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000a2844db pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664382848:12345-0@lo:400:kworker/u8:1.0 Request processed in 39us (300us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189646.527120:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10157 00000100:00100000:3.0:1776189646.527134:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10158 00000100:00100000:3.0:1776189646.527137:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000605c892b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664383296:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189646.527157:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000605c892b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664383296:12345-0@lo:400:kworker/u8:1.0 Request processed in 20us (317us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189646.527162:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10158 00000100:00100000:3.0:1776189646.527193:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57521 00000100:00100000:3.0:1776189646.527196:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000087888c88 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664382912:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189646.527204:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189646.527223:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000087888c88 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664382912:12345-0@lo:38:kworker/u8:1.0 Request processed in 26us (680us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189646.527228:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57521 00000100:00100000:3.0:1776189646.527238:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57522 00000100:00100000:3.0:1776189646.527241:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c5b17f98 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664383040:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189646.527246:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189646.527263:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c5b17f98 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664383040:12345-0@lo:38:kworker/u8:1.0 Request processed in 21us (661us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189646.527268:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57522 00000100:00100000:3.0:1776189646.527283:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540d168e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664382784:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189646.527309:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189646.527324:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000090d63ac9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664383232:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189646.527346:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189646.527363:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1776189646.527367:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004ac5ba05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664383168:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189646.527386:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189646.527401:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a7066b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664383424:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189646.527420:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189646.527443:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664382784 00000100:00100000:3.0:1776189646.527475:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1557 00000100:00100000:3.0:1776189646.527479:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bffe4051 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664382784:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189646.527510:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bffe4051 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664382784:12345-0@lo:400:kworker/u8:1.0 Request processed in 30us (200us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189646.527516:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1557 00000100:00100000:3.0:1776189646.527541:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664383232 00000100:00100000:3.0:1776189646.527562:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664383168 00000100:00100000:3.0:1776189646.527576:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664383424 00000100:00100000:3.0:1776189646.527591:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8707 00000100:00100000:3.0:1776189646.527594:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000036a45663 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664383232:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189646.527625:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000036a45663 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664383232:12345-0@lo:400:kworker/u8:1.0 Request processed in 29us (278us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189646.527632:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8707 00000100:00100000:3.0:1776189646.527644:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8708 00000100:00100000:3.0:1776189646.527647:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0b45fb4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664383168:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189646.527663:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8709 00000100:00100000:1.0:1776189646.527671:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001ce461df pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664383424:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189646.527674:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0b45fb4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664383168:12345-0@lo:400:kworker/u8:1.0 Request processed in 23us (285us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189646.527681:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8708 00000100:00100000:1.0:1776189646.527719:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001ce461df pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664383424:12345-0@lo:400:kworker/u8:1.0 Request processed in 47us (298us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189646.527724:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004ac5ba05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664383168:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189646.527729:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8709 00000100:00100000:3.0:1776189646.527739:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a7066b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664383424:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189646.527749:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540d168e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664382784:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189646.527759:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000090d63ac9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664383232:0@lo:400:kworker/u8:1.0 00000100:00080000:1.0:1776189646.527813:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189646.527817:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189646.527821:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cf6b5d65 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664382912:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776189646.527833:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189646.527837:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189646.527839:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ec15c7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664383040:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776189646.527846:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189646.527848:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189646.527851:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000013c0eaff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664383104:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189646.527870:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189646.528000:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776189646.528011:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000071348c99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664383488:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189646.528019:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776189646.528030:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000114609f6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664383552:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189646.528051:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189646.534635:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664382976 00000100:00100000:1.0:1776189646.534707:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10159 00000100:00100000:1.0:1776189646.534713:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005788f021 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664382976:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189646.537039:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003f67f1c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664383360:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189646.537105:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189646.537142:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dd5e571c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664382848:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189646.537154:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000026520930 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664383296:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189646.537214:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664383360 00000100:00100000:2.0:1776189646.537258:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8710 00000100:00100000:2.0:1776189646.537263:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e071bb1b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664383360:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189646.537308:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e071bb1b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664383360:12345-0@lo:400:kworker/u8:1.0 Request processed in 43us (204us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189646.537315:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8710 00000100:00100000:2.0:1776189646.537345:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003f67f1c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664383360:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189646.537383:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664383488 00000100:00100000:2.0:1776189646.537405:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6998 00000100:00100000:2.0:1776189646.537409:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c3185a52 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664383488:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189646.537423:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189646.537450:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c3185a52 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664383488:12345-0@lo:13:osp-pre-0-1.0 Request processed in 41us (9400us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189646.537457:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6998 00000100:00100000:2.0:1776189646.537490:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189646.537513:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664383552 00000100:00100000:2.0:1776189646.537531:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6999 00000100:00100000:2.0:1776189646.537534:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009b5045f9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664383552:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189646.537544:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189646.537571:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009b5045f9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664383552:12345-0@lo:13:osp-pre-1-1.0 Request processed in 36us (81us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189646.538112:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000114609f6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664383552:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189646.538792:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6999 00000100:00100000:1.0:1776189646.540819:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005788f021 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664382976:12345-0@lo:400:kworker/u8:1.0 Request processed in 6099us (13937us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189646.540835:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10159 00000100:00100000:2.0:1776189646.541036:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004dd53b72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664382976:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189646.541343:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000071348c99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664383488:0@lo:13:osp-pre-0-1.0 00010000:02020000:2.0:1776189646.564292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189646.820276:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189646.845823:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189647.086017:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189647.340347:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189647.596360:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189647.852512:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189647.868453:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189648.108287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189648.366310:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189648.620349:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189648.876285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189648.892405:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189649.133219:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189649.388780:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189649.644458:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189649.900287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189649.916506:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189650.157032:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189650.299118:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189650.299213:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189650.299217:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654963520 00000100:00100000:1.0:1776189650.299287:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10160 00000100:00100000:1.0:1776189650.299293:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000071348c99 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654963520:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189650.299367:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000071348c99 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654963520:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 73us (256us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189650.299375:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10160 00000100:00100000:2.0:1776189650.299609:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654963456 00000100:00100000:2.0:1776189650.299665:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1558 00000100:00100000:2.0:1776189650.299670:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002aae1640 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654963456:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189650.299715:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002aae1640 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654963456:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 44us (503us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189650.299722:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1558 00000100:00100000:0.0:1776189650.301137:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189650.301855:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654963584 00000100:00100000:3.0:1776189650.301905:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57523 00000100:00100000:3.0:1776189650.301911:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ab3412dc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654963584:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:3.0:1776189650.301927:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776189650.301988:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ab3412dc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654963584:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 76us (858us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189650.301995:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57523 00010000:02020000:0.0:1776189650.412293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189650.622782:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6402bd2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664383616:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189650.622874:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189650.623274:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664383616 00000100:00100000:0.0:1776189650.623334:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7000 00000100:00100000:0.0:1776189650.623342:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000114609f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664383616:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189650.623386:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189650.623436:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000114609f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664383616:12345-0@lo:13:osp-pre-1-0.0 Request processed in 94us (563us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189650.623444:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7000 00000100:00100000:3.0:1776189650.623531:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6402bd2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664383616:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189650.623789:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776189650.623800:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000081598856 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664383680:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189650.623843:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189650.625230:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664383680 00000100:00100000:2.0:1776189650.625294:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7001 00000100:00100000:2.0:1776189650.625300:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000f2a9137 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664383680:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189650.625324:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189650.625404:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000f2a9137 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664383680:12345-0@lo:13:osp-pre-0-0.0 Request processed in 103us (1572us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189650.625412:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7001 00000100:00100000:3.0:1776189650.626116:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000081598856 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664383680:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189650.668283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189650.924330:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189650.941233:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189651.180351:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189651.436772:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189651.644379:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776189651.644390:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776189651.644400:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a38efcb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664383744:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189651.644401:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004dd53b72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664383808:0@lo:13:osp-pre-0-1.0 00000100:00080000:1.0:1776189651.649732:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189651.649740:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189651.649743:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189651.649752:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9793 00000100:00080000:1.0:1776189651.649757:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776189651.649762:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189651.649780:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004ef5af0c x1862459664384000/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189651.649805:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189651.649813:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189651.649815:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189651.649818:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9793 00000100:00080000:1.0:1776189651.649820:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776189651.649823:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189651.649842:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001bdbecc8 x1862459664384128/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189651.649850:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189651.649852:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189651.649853:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189651.649856:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9793 00000100:00080000:1.0:1776189651.649858:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1776189651.649860:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189651.649872:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000081598856 x1862459664384192/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189651.649934:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f1df203a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664384256:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189651.653641:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d1846b23 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664384064:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189651.661101:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189651.661169:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3e4a548 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664383872:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189651.661234:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189651.661249:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c90bb74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664384320:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189651.661273:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189651.661322:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664383808 00000100:00100000:3.0:1776189651.661375:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7002 00000100:00100000:3.0:1776189651.661380:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd5e571c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664383808:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776189651.661404:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189651.661443:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd5e571c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664383808:12345-0@lo:13:osp-pre-0-1.0 Request processed in 62us (349us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189651.661450:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7002 00000100:00100000:3.0:1776189651.661472:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664383872 00000100:00100000:3.0:1776189651.661494:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1559 00000100:00100000:3.0:1776189651.661498:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006f66ddc9 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664383872:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189651.661526:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006f66ddc9 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664383872:12345-0@lo:400:kworker/u8:1.0 Request processed in 27us (293us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189651.661533:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1559 00000100:00100000:3.0:1776189651.661560:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664384320 00000100:00100000:3.0:1776189651.661583:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8711 00000100:00100000:3.0:1776189651.661587:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cf6b5d65 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664384320:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189651.661615:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cf6b5d65 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664384320:12345-0@lo:400:kworker/u8:1.0 Request processed in 27us (341us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189651.661620:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8711 00000100:00100000:3.0:1776189651.661742:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3e4a548 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664383872:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189651.661756:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c90bb74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664384320:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189651.661771:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004dd53b72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664383808:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189651.661790:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189651.661810:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664383744 00000100:00100000:0.0:1776189651.661822:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000060733ebe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664383936:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189651.661834:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7003 00000100:00100000:3.0:1776189651.661839:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b25bee14 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664383744:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189651.661851:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189651.661855:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189651.661878:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000431c2394 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664384384:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189651.661893:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b25bee14 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664383744:12345-0@lo:13:osp-pre-1-1.0 Request processed in 52us (105us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189651.661900:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7003 00000100:00100000:0.0:1776189651.661906:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189651.661929:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664383936 00000100:00100000:0.0:1776189651.661939:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a38efcb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664383744:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189651.661957:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664384384 00000100:00100000:0.0:1776189651.661976:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10161 00000100:00100000:0.0:1776189651.661981:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001aa39bf8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664383936:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189651.661985:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10162 00000100:00100000:3.0:1776189651.661992:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000aecac7d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664384384:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189651.662026:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000aecac7d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664384384:12345-0@lo:400:kworker/u8:1.0 Request processed in 34us (121us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189651.662032:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10162 00000100:00100000:3.0:1776189651.662061:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000431c2394 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664384384:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189651.662127:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001aa39bf8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664383936:12345-0@lo:400:kworker/u8:1.0 Request processed in 144us (273us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189651.662134:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10161 00000100:00100000:0.0:1776189651.662203:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000060733ebe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664383936:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189651.662370:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004ef5af0c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664384000:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189651.662399:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189651.662415:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001bdbecc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664384128:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189651.662440:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189651.662452:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000081598856 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664384192:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189651.662474:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189651.662501:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664384000 00000100:00100000:3.0:1776189651.662517:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664384128 00000100:00100000:3.0:1776189651.662538:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57524 00000100:00100000:3.0:1776189651.662541:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004dd53b72 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664384000:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189651.662554:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189651.662587:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004dd53b72 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664384000:12345-0@lo:38:kworker/u8:1.0 Request processed in 44us (187us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189651.662593:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57524 00000100:00100000:3.0:1776189651.662604:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57525 00000100:00100000:3.0:1776189651.662607:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000540d168e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664384128:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189651.662613:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189651.662637:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000540d168e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664384128:12345-0@lo:38:kworker/u8:1.0 Request processed in 29us (197us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189651.662642:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57525 00000100:00100000:3.0:1776189651.662658:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664384192 00000100:00100000:3.0:1776189651.662673:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9174 00000100:00100000:3.0:1776189651.662676:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003f67f1c5 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664384192:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189651.662683:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189651.662713:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003f67f1c5 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664384192:12345-0@lo:38:kworker/u8:1.0 Request processed in 34us (238us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189651.662718:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9174 00000100:00080000:3.0:1776189651.662745:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189651.662749:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189651.662753:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004ef5af0c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664384000:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776189651.662760:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189651.662763:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189651.662765:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001bdbecc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664384128:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776189651.662772:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189651.662774:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189651.662776:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000081598856 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664384192:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189651.666899:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189651.666946:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000545eaa39 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664384512:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189651.666983:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189651.667035:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664384256 00000100:00100000:3.0:1776189651.667081:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664384512 00000100:00100000:3.0:1776189651.667106:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8712 00000100:00100000:3.0:1776189651.667112:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000081598856 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664384256:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189651.667195:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000081598856 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664384256:12345-0@lo:400:kworker/u8:1.0 Request processed in 82us (303us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189651.667203:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8712 00000100:00100000:3.0:1776189651.667220:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8713 00000100:00100000:3.0:1776189651.667224:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001bdbecc8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664384512:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189651.667250:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001bdbecc8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664384512:12345-0@lo:400:kworker/u8:1.0 Request processed in 26us (268us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189651.667256:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8713 00000100:00100000:1.0:1776189651.667676:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f1df203a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664384256:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189651.667688:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000545eaa39 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664384512:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189651.679672:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189651.679717:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009e8b9a01 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664384448:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189651.679750:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189651.679827:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664384064 00000100:00100000:1.0:1776189651.679839:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664384448 00000100:00100000:1.0:1776189651.679870:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8714 00000100:00100000:3.0:1776189651.679871:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10163 00000100:00100000:1.0:1776189651.679875:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a969b6e4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664384448:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189651.679876:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001b0d11fc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664384064:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189651.679981:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001b0d11fc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664384064:12345-0@lo:400:kworker/u8:1.0 Request processed in 104us (315us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189651.679989:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10163 00000100:00100000:1.0:1776189651.682863:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a969b6e4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664384448:12345-0@lo:400:kworker/u8:1.0 Request processed in 2978us (3103us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189651.682881:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8714 00000100:00100000:2.0:1776189651.685342:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d1846b23 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664384064:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189651.685384:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009e8b9a01 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664384448:0@lo:400:kworker/u8:1.0 00010000:02020000:2.0:1776189651.695768:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189651.948311:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189651.964698:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189652.204283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189652.460287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189652.716283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189652.972235:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189652.989688:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189653.228278:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189653.485996:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189653.740814:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189653.998983:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189654.012370:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189654.254171:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189654.509496:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189654.764592:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189655.020300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189655.036766:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189655.276771:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189655.425784:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189655.425835:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189655.425873:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189655.425963:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654963648 00000100:00100000:0.0:1776189655.426027:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1560 00000100:00100000:0.0:1776189655.426033:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000002b5a0b3 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654963648:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189655.426098:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000002b5a0b3 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654963648:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 64us (263us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189655.426108:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1560 00000100:00100000:2.0:1776189655.427162:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654963712 00000100:00100000:2.0:1776189655.427236:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654963776 00000100:00100000:2.0:1776189655.427266:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10164 00000100:00100000:2.0:1776189655.427279:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000545eaa39 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654963712:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189655.427347:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000545eaa39 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654963712:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 67us (1571us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189655.427356:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10164 00000100:00100000:2.0:1776189655.427373:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57526 00000100:00100000:2.0:1776189655.427377:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f1df203a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654963776:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776189655.427391:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189655.427416:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f1df203a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654963776:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 38us (1544us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189655.427422:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57526 00010000:02020000:2.0:1776189655.532992:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189655.742551:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8bc9a84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664384576:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189655.742623:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189655.743211:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664384576 00000100:00100000:0.0:1776189655.743273:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7004 00000100:00100000:0.0:1776189655.743279:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000026520930 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664384576:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189655.743310:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189655.743361:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000026520930 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664384576:12345-0@lo:13:osp-pre-1-0.0 Request processed in 81us (739us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189655.743367:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7004 00000100:00100000:0.0:1776189655.743414:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8bc9a84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664384576:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189655.743570:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004ef5af0c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664384640:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189655.743598:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189655.744163:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664384640 00000100:00100000:1.0:1776189655.744229:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7005 00000100:00100000:1.0:1776189655.744233:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b8bc9a84 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664384640:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189655.744253:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189655.744343:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b8bc9a84 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664384640:12345-0@lo:13:osp-pre-0-0.0 Request processed in 109us (744us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189655.744350:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7005 00000100:00100000:2.0:1776189655.744426:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004ef5af0c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664384640:0@lo:13:osp-pre-0-0.0 00010000:02020000:2.0:1776189655.789096:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189656.044890:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189656.060450:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189656.300294:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189656.557867:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189656.764484:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776189656.764506:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000090d63ac9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664384704:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189656.764589:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189656.764641:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664384704 00000100:00100000:0.0:1776189656.764682:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7006 00000100:00100000:0.0:1776189656.764688:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053ddeb10 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664384704:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776189656.764721:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189656.764760:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053ddeb10 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664384704:12345-0@lo:13:osp-pre-0-1.0 Request processed in 71us (172us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189656.764767:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7006 00000100:00100000:0.0:1776189656.764840:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000090d63ac9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664384704:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189656.765304:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776189656.765317:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a7066b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664384768:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189656.765349:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189656.765428:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664384768 00000100:00100000:3.0:1776189656.765466:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7007 00000100:00100000:3.0:1776189656.765470:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000087513f3f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664384768:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189656.765487:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189656.765524:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000087513f3f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664384768:12345-0@lo:13:osp-pre-1-1.0 Request processed in 53us (175us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189656.765530:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7007 00000100:00100000:0.0:1776189656.765998:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a7066b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664384768:0@lo:13:osp-pre-1-1.0 00000100:00080000:3.0:1776189656.766239:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189656.766255:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189656.766258:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189656.766262:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9798 00000100:00080000:3.0:1776189656.766265:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1776189656.766269:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189656.766289:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004ac5ba05 x1862459664384960/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189656.766314:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189656.766318:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189656.766319:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189656.766322:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9798 00000100:00080000:3.0:1776189656.766324:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1776189656.766327:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189656.766338:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000797ba66f x1862459664385088/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189656.766346:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189656.766348:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189656.766349:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189656.766352:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9798 00000100:00080000:3.0:1776189656.766354:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1776189656.766357:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189656.766367:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b7713fdb x1862459664385152/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189656.766401:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004ac5ba05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664384960:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189656.766438:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189656.766454:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000797ba66f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664385088:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189656.766475:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189656.766485:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b7713fdb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664385152:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189656.766504:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189656.766535:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664385152 00000100:00100000:2.0:1776189656.766672:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000073262ad2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664385024:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189656.766710:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189656.766730:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:2.0:1776189656.766734:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000090d63ac9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664384896:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189656.766754:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189656.766767:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000df4082ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664385344:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189656.766787:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189656.766811:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664384960 00000100:00100000:2.0:1776189656.766832:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664385088 00000100:00100000:2.0:1776189656.766839:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664385024 00000100:00100000:2.0:1776189656.766853:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664384896 00000100:00100000:2.0:1776189656.766864:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664385344 00000100:00100000:2.0:1776189656.766875:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10165 00000100:00100000:2.0:1776189656.766879:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000375d4d18 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664385024:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189656.766925:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000375d4d18 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664385024:12345-0@lo:400:kworker/u8:1.0 Request processed in 44us (214us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189656.766931:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10165 00000100:00100000:2.0:1776189656.766945:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10166 00000100:00100000:2.0:1776189656.766948:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c921b786 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664384896:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189656.766973:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c921b786 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664384896:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (219us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189656.766978:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10166 00000100:00100000:2.0:1776189656.766989:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10167 00000100:00100000:2.0:1776189656.766992:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000023fe30ce pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664385344:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189656.767013:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000023fe30ce pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664385344:12345-0@lo:400:kworker/u8:1.0 Request processed in 20us (226us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189656.767017:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10167 00000100:00100000:2.0:1776189656.767028:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57527 00000100:00100000:2.0:1776189656.767030:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b4a5ce06 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664384960:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189656.767043:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189656.767072:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b4a5ce06 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664384960:12345-0@lo:38:kworker/u8:1.0 Request processed in 40us (634us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189656.767078:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57527 00000100:00100000:2.0:1776189656.767089:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57528 00000100:00100000:2.0:1776189656.767092:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000075d6a9fc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664385088:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189656.767098:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189656.767118:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000075d6a9fc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664385088:12345-0@lo:38:kworker/u8:1.0 Request processed in 25us (642us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189656.767123:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57528 00000100:00080000:2.0:1776189656.767157:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189656.767161:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189656.767164:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004ac5ba05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664384960:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189656.768900:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000090d63ac9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664384896:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189656.768916:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000df4082ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664385344:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189656.768927:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000073262ad2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664385024:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189656.768960:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000308dffba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664385216:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189656.769010:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189656.769035:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1776189656.769041:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a7066b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664384832:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189656.769070:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189656.769084:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b9ada007 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664385280:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189656.769102:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189656.769132:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664384832 00000100:00100000:1.0:1776189656.769168:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1561 00000100:00100000:1.0:1776189656.769195:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000064f5974 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664384832:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189656.769238:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000064f5974 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664384832:12345-0@lo:400:kworker/u8:1.0 Request processed in 64us (168us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189656.769257:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1561 00000100:00100000:1.0:1776189656.769292:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664385216 00000100:00100000:1.0:1776189656.769319:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664385280 00000100:00100000:3.0:1776189656.769337:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9175 00000100:00100000:1.0:1776189656.769343:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8715 00000100:00100000:3.0:1776189656.769346:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001fc79665 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664385152:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189656.769351:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000060733ebe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664385216:12345-0@lo:400:kworker/u8:1.0 00010000:02000000:3.0:1776189656.769363:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189656.769389:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000060733ebe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664385216:12345-0@lo:400:kworker/u8:1.0 Request processed in 37us (378us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189656.769397:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8715 00000100:00100000:1.0:1776189656.769410:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8716 00000100:00100000:1.0:1776189656.769413:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000431c2394 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664385280:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189656.769437:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000431c2394 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664385280:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (335us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189656.769448:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8716 00000100:00100000:1.0:1776189656.769470:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a7066b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664384832:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189656.769480:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b9ada007 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664385280:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189656.769487:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000308dffba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664385216:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189656.770854:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001fc79665 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664385152:12345-0@lo:38:kworker/u8:1.0 Request processed in 1505us (4344us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189656.770870:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9175 00000100:00080000:2.0:1776189656.771231:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189656.771236:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189656.771252:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000797ba66f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664385088:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776189656.771273:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189656.771275:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189656.771278:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b7713fdb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664385152:0@lo:38:kworker/u8:1.0 00010000:02020000:2.0:1776189656.812754:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189657.068493:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189657.084799:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189657.326063:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189657.580319:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189657.836870:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189658.096205:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189658.108751:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189658.348508:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189658.604359:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189658.860300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189659.116865:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189659.132524:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189659.372286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189659.628296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189659.884326:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189660.140290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189660.157862:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189660.397493:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189660.538454:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189660.538507:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189660.538668:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654963840 00000100:00100000:0.0:1776189660.538722:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1562 00000100:00100000:0.0:1776189660.538728:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000084b636aa pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654963840:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1776189660.538805:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000084b636aa pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654963840:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 76us (359us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189660.538813:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1562 00000100:00100000:2.0:1776189660.539049:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654963904 00000100:00100000:2.0:1776189660.539088:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10168 00000100:00100000:2.0:1776189660.539093:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006c90bb74 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654963904:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776189660.539131:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006c90bb74 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2280:x1862459654963904:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 38us (625us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189660.539139:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10168 00000100:00100000:0.0:1776189660.539211:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189660.539650:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654963968 00000100:00100000:2.0:1776189660.539717:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57529 00000100:00100000:2.0:1776189660.539722:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e8ddaf27 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654963968:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:2.0:1776189660.539738:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189660.539786:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e8ddaf27 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654963968:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 63us (577us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189660.539792:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57529 00010000:02020000:2.0:1776189660.652297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189660.860739:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002c2b2af4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664385408:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189660.860829:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189660.861106:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776189660.861109:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664385408 00000100:00100000:1.0:1776189660.861116:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3e4a548 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664385472:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189660.861161:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189660.861227:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664385472 00000100:00100000:1.0:1776189660.861256:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7008 00000100:00100000:1.0:1776189660.861261:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000099b8eb0d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664385408:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189660.861289:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189660.861325:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000099b8eb0d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664385408:12345-0@lo:13:osp-pre-1-0.0 Request processed in 63us (497us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189660.861332:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7008 00000100:00100000:1.0:1776189660.861349:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7009 00000100:00100000:1.0:1776189660.861353:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004ef5af0c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664385472:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189660.861381:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189660.861414:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004ef5af0c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664385472:12345-0@lo:13:osp-pre-0-0.0 Request processed in 60us (262us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189660.861420:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7009 00000100:00100000:0.0:1776189660.861459:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3e4a548 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664385472:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189660.861863:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002c2b2af4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664385408:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189660.908844:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189661.164277:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189661.180485:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189661.420296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189661.676942:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:2.0:1776189661.884924:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189661.884932:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189661.884935:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189661.884942:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9803 00000100:00080000:2.0:1776189661.884946:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776189661.884951:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189661.884969:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004920d098 x1862459664385664/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189661.884970:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace99b9b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664385536:0@lo:400:kworker/u8:0.0 00000100:00080000:2.0:1776189661.884988:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189661.884991:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189661.884992:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189661.884995:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9803 00000100:00080000:2.0:1776189661.884997:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776189661.885000:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189661.885008:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004ac5ba05 x1862459664385792/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189661.885015:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189661.885017:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189661.885019:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189661.885021:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9803 00000100:00080000:2.0:1776189661.885022:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776189661.885025:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189661.885033:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189661.885036:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000b36a9c1 x1862459664385856/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189661.885065:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000073262ad2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664385984:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189661.885086:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000797ba66f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664385920:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189661.885090:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189661.885110:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189661.885122:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664385536 00000100:00100000:2.0:1776189661.885124:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000013c0eaff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664386176:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189661.885193:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1563 00000100:00100000:1.0:1776189661.885198:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c1f77e46 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664385536:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189661.885203:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189661.885228:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000266f5a0c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664385600:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189661.885241:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c1f77e46 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664385536:12345-0@lo:400:kworker/u8:0.0 Request processed in 43us (208us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189661.885248:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1563 00000100:00100000:2.0:1776189661.885264:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189661.885278:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664385984 00000100:00100000:2.0:1776189661.885278:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002fbab827 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664386048:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189661.885297:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189661.885312:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664385920 00000100:00100000:2.0:1776189661.885312:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1776189661.885316:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088ec3431 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664385728:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189661.885328:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664386176 00000100:00100000:2.0:1776189661.885335:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189661.885344:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8717 00000100:00100000:2.0:1776189661.885346:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000df4082ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664386112:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189661.885348:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009e8b9a01 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664385984:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189661.885363:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189661.885373:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009e8b9a01 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664385984:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (282us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189661.885379:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8717 00000100:00100000:2.0:1776189661.885386:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004920d098 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664385664:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189661.885389:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664386112 00000100:00100000:1.0:1776189661.885404:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8718 00000100:00100000:1.0:1776189661.885407:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007ec15c7b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664385920:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189661.885407:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189661.885419:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004ac5ba05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664385792:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189661.885434:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007ec15c7b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664385920:12345-0@lo:400:kworker/u8:0.0 Request processed in 25us (323us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189661.885435:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189661.885439:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8718 00000100:00100000:2.0:1776189661.885444:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000b36a9c1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664385856:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189661.885449:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8719 00000100:00100000:1.0:1776189661.885452:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008a7066b0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664386176:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189661.885464:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189661.885472:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008a7066b0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664386176:12345-0@lo:400:kworker/u8:0.0 Request processed in 20us (269us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189661.885477:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8719 00000100:00100000:1.0:1776189661.885487:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8720 00000100:00100000:1.0:1776189661.885490:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d4382e95 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664386112:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189661.885503:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664385600 00000100:00100000:1.0:1776189661.885518:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d4382e95 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664386112:12345-0@lo:400:kworker/u8:0.0 Request processed in 27us (155us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189661.885523:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8720 00000100:00100000:2.0:1776189661.885525:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664386048 00000100:00100000:2.0:1776189661.885533:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664385728 00000100:00100000:2.0:1776189661.885541:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10169 00000100:00100000:2.0:1776189661.885544:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000090d63ac9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664385600:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189661.885570:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000073262ad2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664385984:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189661.885570:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000090d63ac9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664385600:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (306us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189661.885576:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10169 00000100:00100000:1.0:1776189661.885579:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace99b9b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664385536:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189661.885586:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664385664 00000100:00100000:2.0:1776189661.885594:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664385792 00000100:00100000:2.0:1776189661.885602:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10170 00000100:00100000:2.0:1776189661.885605:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007df90c3e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664386048:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189661.885628:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007df90c3e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664386048:12345-0@lo:400:kworker/u8:0.0 Request processed in 22us (331us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189661.885633:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10170 00000100:00100000:2.0:1776189661.885643:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10171 00000100:00100000:2.0:1776189661.885647:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ee179fa1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664385728:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189661.885668:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ee179fa1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664385728:12345-0@lo:400:kworker/u8:0.0 Request processed in 20us (333us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189661.885673:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10171 00000100:00100000:2.0:1776189661.885683:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57530 00000100:00100000:2.0:1776189661.885686:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005c6d79ae pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664385664:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189661.885698:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189661.885726:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005c6d79ae pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664385664:12345-0@lo:38:kworker/u8:0.0 Request processed in 39us (319us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189661.885732:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57530 00000100:00100000:2.0:1776189661.885740:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57531 00000100:00100000:2.0:1776189661.885743:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a943cb24 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664385792:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189661.885749:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189661.885765:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a943cb24 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664385792:12345-0@lo:38:kworker/u8:0.0 Request processed in 21us (330us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189661.885770:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57531 00000100:00100000:2.0:1776189661.885801:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664385856 00000100:00100000:2.0:1776189661.885817:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9176 00000100:00100000:2.0:1776189661.885820:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000058863251 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664385856:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189661.885827:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189661.885847:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000058863251 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664385856:12345-0@lo:38:kworker/u8:0.0 Request processed in 25us (383us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189661.885852:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9176 00000100:00080000:2.0:1776189661.885878:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189661.885881:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189661.885884:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004920d098 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664385664:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776189661.885891:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189661.885893:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189661.885895:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004ac5ba05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664385792:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776189661.885901:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189661.885903:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189661.885905:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000b36a9c1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664385856:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189661.885933:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000797ba66f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664385920:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189661.885942:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000013c0eaff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664386176:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189661.885963:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088ec3431 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664385728:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189661.885974:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000df4082ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664386112:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189661.885981:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000266f5a0c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664385600:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189661.885988:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002fbab827 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664386048:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189661.886431:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000058c450e1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664386240:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189661.886462:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189661.886523:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664386240 00000100:00100000:3.0:1776189661.886545:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7010 00000100:00100000:3.0:1776189661.886549:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d52de5d9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664386240:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189661.886563:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189661.886624:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d52de5d9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664386240:12345-0@lo:13:osp-pre-1-1.0 Request processed in 74us (161us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189661.886655:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000058c450e1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664386240:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189661.886671:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776189661.886677:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d1846b23 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664386304:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189661.886697:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189661.886721:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664386304 00000100:00100000:3.0:1776189661.889909:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7010 00000100:00100000:2.0:1776189661.889967:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7011 00000100:00100000:2.0:1776189661.889978:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c6bc9cdf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664386304:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189661.890006:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189661.890078:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c6bc9cdf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664386304:12345-0@lo:13:osp-pre-0-1.0 Request processed in 102us (3380us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189661.890085:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7011 00000100:00100000:2.0:1776189661.890163:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d1846b23 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664386304:0@lo:13:osp-pre-0-1.0 00010000:02020000:2.0:1776189661.932402:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189662.188356:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189662.204770:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189662.444342:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189662.700659:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189662.957363:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189663.213058:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189663.230611:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189663.468406:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189663.725032:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189663.980291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189664.236291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189664.252656:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189664.492515:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189664.748477:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189665.004422:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189665.260444:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189665.277199:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189665.523968:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189665.662921:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189665.662973:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189665.663006:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189665.663094:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654964032 00000100:00100000:1.0:1776189665.663153:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1564 00000100:00100000:1.0:1776189665.663159:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f055c03f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654964032:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776189665.663262:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f055c03f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654964032:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 101us (289us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189665.663270:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1564 00000100:00100000:3.0:1776189665.663709:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654964096 00000100:00100000:3.0:1776189665.663747:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654964160 00000100:00100000:3.0:1776189665.663772:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10172 00000100:00100000:3.0:1776189665.663777:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007c812403 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654964096:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1776189665.663814:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007c812403 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654964096:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 36us (901us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189665.663821:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10172 00000100:00100000:3.0:1776189665.663834:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57532 00000100:00100000:3.0:1776189665.663837:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002c2b2af4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654964160:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:3.0:1776189665.663851:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776189665.663876:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002c2b2af4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654964160:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 37us (870us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189665.663882:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57532 00010000:02020000:2.0:1776189665.780305:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189665.980757:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d1846b23 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664386368:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189665.980824:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189665.980885:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664386368 00000100:00100000:3.0:1776189665.980940:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7012 00000100:00100000:3.0:1776189665.980946:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e6402bd2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664386368:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189665.980977:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189665.981033:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e6402bd2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664386368:12345-0@lo:13:osp-pre-0-0.0 Request processed in 86us (209us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189665.981040:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7012 00000100:00100000:0.0:1776189665.981289:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d1846b23 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664386368:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189665.981553:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776189665.981562:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a38efcb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664386432:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189665.981593:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189665.981685:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664386432 00000100:00100000:1.0:1776189665.981723:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7013 00000100:00100000:1.0:1776189665.981728:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c27afb9e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664386432:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189665.981749:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189665.981799:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c27afb9e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664386432:12345-0@lo:13:osp-pre-1-0.0 Request processed in 69us (205us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189665.981806:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7013 00000100:00100000:0.0:1776189665.982055:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a38efcb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664386432:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189666.037346:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189666.292341:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189666.303903:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189666.550323:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189666.804280:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:2.0:1776189667.004519:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189667.004526:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189667.004528:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189667.004536:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9808 00000100:00080000:2.0:1776189667.004539:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776189667.004544:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189667.004654:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002fbab827 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664386496:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189667.004718:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189667.004768:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664386496 00000100:00100000:0.0:1776189667.004807:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1565 00000100:00100000:0.0:1776189667.004812:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000353f317b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664386496:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189667.004859:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000353f317b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664386496:12345-0@lo:400:kworker/u8:1.0 Request processed in 47us (142us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189667.004866:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1565 00000100:00100000:0.0:1776189667.004907:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002fbab827 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664386496:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189667.004941:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000df4082ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664386560:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189667.004967:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189667.005015:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664386560 00000100:00100000:0.0:1776189667.005050:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10173 00000100:00100000:0.0:1776189667.005054:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c68cc49e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664386560:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189667.005090:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c68cc49e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664386560:12345-0@lo:400:kworker/u8:1.0 Request processed in 35us (123us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189667.005096:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10173 00000100:00100000:0.0:1776189667.005121:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000df4082ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664386560:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189667.005246:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776189667.005254:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5f1e324 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664386624:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189667.005281:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189667.005303:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664386624 00000100:00100000:3.0:1776189667.005328:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7014 00000100:00100000:3.0:1776189667.005332:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000058c450e1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664386624:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189667.005355:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189667.005387:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000058c450e1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664386624:12345-0@lo:13:osp-pre-1-1.0 Request processed in 54us (106us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189667.005393:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7014 00000100:00100000:3.0:1776189667.005420:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5f1e324 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664386624:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189667.005533:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776189667.005542:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3e4a548 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664386688:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189667.005581:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189667.005606:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664386688 00000100:00100000:1.0:1776189667.005628:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7015 00000100:00100000:1.0:1776189667.005631:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d1846b23 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664386688:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189667.005643:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189667.005668:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d1846b23 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664386688:12345-0@lo:13:osp-pre-0-1.0 Request processed in 37us (88us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189667.005674:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7015 00000100:00100000:1.0:1776189667.005705:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3e4a548 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664386688:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189667.005927:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000df4082ad x1862459664386752/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189667.005976:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189667.005980:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189667.005982:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189667.005985:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9808 00000100:00080000:2.0:1776189667.005987:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776189667.005991:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189667.006003:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000b36a9c1 x1862459664386880/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189667.006011:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189667.006014:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189667.006015:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189667.006017:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9808 00000100:00080000:2.0:1776189667.006033:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776189667.006037:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189667.006048:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004ac5ba05 x1862459664386944/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189667.006118:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a388dd2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664387072:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189667.006149:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189667.006168:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776189667.006191:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000761163f9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664387008:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189667.006214:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189667.006240:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664387072 00000100:00100000:2.0:1776189667.006264:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664387008 00000100:00100000:2.0:1776189667.006279:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8721 00000100:00100000:2.0:1776189667.006282:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000085a66d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664387072:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189667.006318:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000085a66d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664387072:12345-0@lo:400:kworker/u8:1.0 Request processed in 34us (168us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189667.006324:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8721 00000100:00100000:2.0:1776189667.006336:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8722 00000100:00100000:2.0:1776189667.006340:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000b321ca3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664387008:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189667.006364:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000b321ca3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664387008:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (150us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189667.006369:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8722 00000100:00100000:2.0:1776189667.006403:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000761163f9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664387008:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189667.006411:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a388dd2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664387072:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189667.006483:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a02f1c47 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664387136:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189667.006511:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189667.006534:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776189667.006539:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000013c0eaff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664386816:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189667.006583:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189667.006611:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664387136 00000100:00100000:3.0:1776189667.006633:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664386816 00000100:00100000:3.0:1776189667.006650:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10174 00000100:00100000:3.0:1776189667.006653:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b5f1e324 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664387136:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189667.006683:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b5f1e324 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664387136:12345-0@lo:400:kworker/u8:1.0 Request processed in 29us (172us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189667.006689:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10174 00000100:00100000:3.0:1776189667.006700:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10175 00000100:00100000:3.0:1776189667.006703:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c794d022 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664386816:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189667.006726:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c794d022 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664386816:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (144us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189667.006730:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10175 00000100:00100000:3.0:1776189667.006760:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000013c0eaff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664386816:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189667.006767:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a02f1c47 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664387136:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189667.006922:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000df4082ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664386752:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189667.006951:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189667.006965:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000b36a9c1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664386880:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189667.006982:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189667.006992:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004ac5ba05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664386944:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189667.007010:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189667.008729:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664386944 00000100:00100000:2.0:1776189667.008791:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9177 00000100:00100000:2.0:1776189667.008796:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f8dcf939 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664386944:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189667.008814:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189667.008893:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f8dcf939 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664386944:12345-0@lo:38:kworker/u8:1.0 Request processed in 95us (1882us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189667.008900:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9177 00000100:00100000:2.0:1776189667.008927:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664386752 00000100:00100000:2.0:1776189667.008943:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664386880 00000100:00100000:2.0:1776189667.008960:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57533 00000100:00100000:2.0:1776189667.008964:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000038a7e045 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664386752:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189667.008971:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189667.008995:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000038a7e045 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664386752:12345-0@lo:38:kworker/u8:1.0 Request processed in 31us (2045us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189667.009001:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57533 00000100:00100000:2.0:1776189667.009010:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57534 00000100:00100000:2.0:1776189667.009013:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000838deece pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664386880:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189667.009029:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189667.009051:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000838deece pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664386880:12345-0@lo:38:kworker/u8:1.0 Request processed in 36us (2068us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189667.009056:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57534 00000100:00080000:1.0:1776189667.012726:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189667.012737:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189667.012744:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000df4082ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664386752:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776189667.012757:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189667.012760:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189667.012762:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000b36a9c1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664386880:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776189667.012772:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189667.012774:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189667.012776:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004ac5ba05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664386944:0@lo:38:kworker/u8:1.0 00010000:02020000:2.0:1776189667.060861:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189667.316694:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189667.324493:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189667.572346:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189667.828986:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189668.086758:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189668.340306:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189668.348442:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189668.596295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189668.852945:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189669.108290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189669.364298:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189669.376608:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189669.620350:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189669.876635:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189670.134420:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189670.389301:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189670.396396:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189670.644385:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189670.782108:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189670.782166:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189670.782229:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654964288 00000100:00100000:3.0:1776189670.782278:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10176 00000100:00100000:0.0:1776189670.782284:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189670.782288:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002fbab827 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654964288:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1776189670.782332:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654964352 00000100:00100000:3.0:1776189670.782366:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002fbab827 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654964288:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 78us (264us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189670.782391:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57535 00000100:00100000:0.0:1776189670.782398:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a38efcb6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654964352:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:0.0:1776189670.782415:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189670.782447:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a38efcb6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654964352:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 49us (167us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189670.782501:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654964224 00000100:00100000:1.0:1776189670.782536:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1566 00000100:00100000:1.0:1776189670.782541:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003c5b81e6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654964224:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776189670.782584:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003c5b81e6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654964224:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 42us (419us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189670.782590:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1566 00000100:00100000:3.0:1776189670.787216:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10176 00000100:00100000:0.0:1776189670.788085:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57535 00010000:02020000:0.0:1776189670.901655:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189671.101289:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776189671.101339:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776189671.101361:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000093b814ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664387264:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189671.101434:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189671.101618:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664387264 00000100:00100000:0.0:1776189671.101668:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7016 00000100:00100000:0.0:1776189671.101673:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c22b559c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664387264:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189671.101699:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189671.101738:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c22b559c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664387264:12345-0@lo:13:osp-pre-1-0.0 Request processed in 64us (305us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189671.101744:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7016 00000100:00100000:3.0:1776189671.101974:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000093b814ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664387264:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189671.102226:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3e4a548 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664387200:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189671.102259:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189671.102336:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664387200 00000100:00100000:3.0:1776189671.102369:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7017 00000100:00100000:3.0:1776189671.102374:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004ac5ba05 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664387200:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189671.102392:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189671.102442:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004ac5ba05 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664387200:12345-0@lo:13:osp-pre-0-0.0 Request processed in 68us (183us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189671.102449:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7017 00000100:00100000:1.0:1776189671.102679:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3e4a548 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664387200:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189671.156375:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189671.412599:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189671.420866:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189671.668331:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189671.924341:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:2.0:1776189672.124679:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189672.124688:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189672.124691:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189672.124698:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9813 00000100:00080000:2.0:1776189672.124701:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776189672.124706:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189672.124726:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bc501d10 x1862459664387456/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189672.124750:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189672.124753:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189672.124754:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189672.124758:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9813 00000100:00080000:2.0:1776189672.124759:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776189672.124762:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189672.124771:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b9d589b6 x1862459664387584/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189672.124777:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189672.124780:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189672.124781:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189672.124784:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9813 00000100:00080000:2.0:1776189672.124785:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776189672.124788:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189672.124797:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a7368321 x1862459664387648/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189672.124856:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bc501d10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664387456:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189672.124930:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189672.125469:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a388dd2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664387328:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189672.125540:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664387456 00000100:00100000:3.0:1776189672.125576:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000761163f9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664387392:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189672.125580:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57536 00000100:00100000:0.0:1776189672.125585:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007390d733 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664387456:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189672.125607:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189672.125820:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b9d589b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664387584:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189672.133844:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189672.133882:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000053d4c624 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664387776:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189672.134144:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189672.134580:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007390d733 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664387456:12345-0@lo:38:kworker/u8:1.0 Request processed in 8992us (9648us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189672.134591:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57536 00000100:00100000:3.0:1776189672.134668:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e13527ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664387840:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189672.134697:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189672.134719:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1776189672.134724:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002ea4a019 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664387520:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189672.134740:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189672.134752:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008870952c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664387904:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189672.134770:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189672.134805:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664387392 00000100:00100000:3.0:1776189672.134835:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664387840 00000100:00100000:3.0:1776189672.134844:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664387520 00000100:00100000:3.0:1776189672.134855:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10177 00000100:00100000:3.0:1776189672.134859:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000de589d43 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664387392:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189672.134924:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000de589d43 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664387392:12345-0@lo:400:kworker/u8:1.0 Request processed in 64us (782us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189672.134931:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10177 00000100:00100000:3.0:1776189672.134952:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10178 00000100:00100000:3.0:1776189672.134955:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000991081bf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664387840:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189672.134982:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000991081bf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664387840:12345-0@lo:400:kworker/u8:1.0 Request processed in 25us (284us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189672.134987:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10178 00000100:00100000:3.0:1776189672.135002:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10179 00000100:00100000:3.0:1776189672.135005:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f05a9c16 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664387520:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189672.135028:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f05a9c16 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664387520:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (288us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189672.135034:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10179 00000100:00100000:0.0:1776189672.135054:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006ad1d61a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664387712:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189672.135071:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664387904 00000100:00100000:0.0:1776189672.135090:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189672.135114:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000080b4f511 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664387968:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189672.135145:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189672.135226:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664387328 00000100:00100000:1.0:1776189672.135250:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1567 00000100:00100000:1.0:1776189672.135262:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008e1954f2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664387328:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189672.135293:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008e1954f2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664387328:12345-0@lo:400:kworker/u8:1.0 Request processed in 39us (1456us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189672.135299:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1567 00000100:00100000:3.0:1776189672.135369:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664387712 00000100:00100000:3.0:1776189672.135394:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664387968 00000100:00100000:3.0:1776189672.135413:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8723 00000100:00100000:0.0:1776189672.135415:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776189672.135418:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000024796ea pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664387904:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189672.135425:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002126a7bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664388032:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189672.135455:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000024796ea pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664387904:12345-0@lo:400:kworker/u8:1.0 Request processed in 37us (684us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189672.135465:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189672.135465:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8723 00000100:00100000:3.0:1776189672.135478:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8724 00000100:00100000:3.0:1776189672.135482:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007a0ce2a1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664387712:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189672.135502:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664388032 00000100:00100000:3.0:1776189672.135511:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007a0ce2a1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664387712:12345-0@lo:400:kworker/u8:1.0 Request processed in 28us (422us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189672.135517:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8724 00000100:00100000:3.0:1776189672.135528:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8725 00000100:00100000:3.0:1776189672.135534:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000067bb60df pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+6:3363:x1862459664387968:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189672.135536:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7018 00000100:00100000:0.0:1776189672.135544:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000076bff338 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3363:x1862459664388032:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189672.135560:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189672.135565:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000067bb60df pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+6:3363:x1862459664387968:12345-0@lo:400:kworker/u8:1.0 Request processed in 31us (419us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189672.135571:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8725 00000100:00100000:0.0:1776189672.135597:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000076bff338 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664388032:12345-0@lo:13:osp-pre-1-1.0 Request processed in 52us (134us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189672.135604:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7018 00000100:00100000:3.0:1776189672.135624:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3e4a548 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664388096:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189672.135650:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189672.135680:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002ea4a019 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664387520:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189672.135689:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008870952c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664387904:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189672.135697:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000761163f9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664387392:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189672.135705:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e13527ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664387840:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189672.135739:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664388096 00000100:00100000:3.0:1776189672.135756:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7019 00000100:00100000:3.0:1776189672.135760:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000bb8b061 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664388096:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776189672.135770:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189672.135794:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000bb8b061 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664388096:12345-0@lo:13:osp-pre-0-1.0 Request processed in 34us (144us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189672.135799:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7019 00000100:00100000:3.0:1776189672.135821:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3e4a548 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664388096:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189672.136526:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002126a7bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664388032:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189672.136549:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006ad1d61a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664387712:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189672.136557:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000080b4f511 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664387968:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189672.136604:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189672.136632:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a388dd2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664387328:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189672.136898:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664387776 00000100:00100000:3.0:1776189672.136943:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8726 00000100:00100000:3.0:1776189672.136947:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000070105d1b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664387776:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189672.136991:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000070105d1b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664387776:12345-0@lo:400:kworker/u8:1.0 Request processed in 43us (386us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189672.136997:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8726 00000100:00100000:0.0:1776189672.137067:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000053d4c624 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664387776:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189672.140194:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189672.140241:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a7368321 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664387648:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189672.140280:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1776189672.140348:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189672.140352:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189672.140356:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bc501d10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664387456:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189672.140408:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664387584 00000100:00100000:3.0:1776189672.140499:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57537 00000100:00100000:3.0:1776189672.140506:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dc4a257b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664387584:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189672.140523:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189672.140570:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dc4a257b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664387584:12345-0@lo:38:kworker/u8:1.0 Request processed in 63us (400us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189672.140580:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57537 00000100:00080000:2.0:1776189672.140627:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189672.140631:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189672.140635:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b9d589b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664387584:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189672.141233:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664387648 00000100:00100000:0.0:1776189672.141270:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9178 00000100:00100000:0.0:1776189672.141275:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e0409ecf pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664387648:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189672.141287:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189672.141339:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e0409ecf pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664387648:12345-0@lo:38:kworker/u8:1.0 Request processed in 63us (1059us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189672.141345:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9178 00000100:00080000:3.0:1776189672.144372:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189672.144384:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189672.144391:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a7368321 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664387648:0@lo:38:kworker/u8:1.0 00010000:02020000:0.0:1776189672.181923:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189672.436287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189672.449126:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189672.693265:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189672.948250:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189673.204297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189673.460287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189673.468409:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189673.716798:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189673.972298:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189674.230503:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189674.484283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189674.492513:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189674.740394:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189674.996672:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189675.252308:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189675.508795:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189675.517299:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189675.764520:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189675.899125:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189675.899169:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189675.899256:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189675.899568:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654964480 00000100:00100000:2.0:1776189675.899623:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654964544 00000100:00100000:2.0:1776189675.899647:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10180 00000100:00100000:2.0:1776189675.899653:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000291d219d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654964480:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189675.899735:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000291d219d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654964480:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 81us (629us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189675.899743:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10180 00000100:00100000:1.0:1776189675.899770:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57538 00000100:00100000:1.0:1776189675.899778:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053eca24c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654964544:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00000100:00100000:2.0:1776189675.899789:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654964416 00010000:02000000:1.0:1776189675.899794:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189675.899820:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1568 00000100:00100000:2.0:1776189675.899825:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001aaffad0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654964416:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189675.900451:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053eca24c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654964544:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 671us (1280us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189675.900460:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57538 00000100:00100000:2.0:1776189675.903921:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001aaffad0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654964416:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 4085us (4656us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189675.903942:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1568 00010000:02020000:0.0:1776189676.020950:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189676.220564:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ab0cd203 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664388160:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189676.220633:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189676.220755:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664388160 00000100:00100000:2.0:1776189676.220851:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7020 00000100:00100000:2.0:1776189676.220856:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000043ae8a46 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664388160:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189676.220886:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189676.220937:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000043ae8a46 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664388160:12345-0@lo:13:osp-pre-1-0.0 Request processed in 80us (305us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189676.220945:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7020 00000100:00100000:0.0:1776189676.221019:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ab0cd203 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664388160:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189676.223001:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776189676.223020:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ab0cd203 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664388224:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189676.223077:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189676.230834:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664388224 00000100:00100000:3.0:1776189676.230899:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7021 00000100:00100000:3.0:1776189676.230905:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f1c366f3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664388224:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189676.230938:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189676.231003:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f1c366f3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664388224:12345-0@lo:13:osp-pre-0-0.0 Request processed in 97us (7926us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189676.231010:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7021 00000100:00100000:3.0:1776189676.231062:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ab0cd203 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664388224:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189676.276384:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189676.532962:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189676.540922:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189676.788285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189677.044468:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189677.244390:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b9d589b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664388288:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189677.244462:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1776189677.244531:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189677.244536:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189677.244539:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189677.244543:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9819 00000100:00080000:2.0:1776189677.244547:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00080000:2.0:1776189677.244551:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189677.244569:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d3fa5d0f x1862459664388480/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189677.244633:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8a51369 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664388544:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189677.244667:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189677.244706:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3e4a548 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664388352:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189677.244712:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664388544 00000100:00100000:0.0:1776189677.244746:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189677.244755:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10181 00000100:00100000:1.0:1776189677.244762:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000b36a9c1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664388544:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189677.244771:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776189677.244779:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bc501d10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664388416:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189677.244826:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189677.244836:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000b36a9c1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664388544:12345-0@lo:400:kworker/u8:1.0 Request processed in 73us (168us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189677.244844:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10181 00000100:00100000:1.0:1776189677.244864:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664388416 00000100:00100000:1.0:1776189677.244902:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10182 00000100:00100000:1.0:1776189677.244905:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001ca51788 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664388416:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189677.244937:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001ca51788 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664388416:12345-0@lo:400:kworker/u8:1.0 Request processed in 30us (112us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189677.244943:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10182 00000100:00100000:1.0:1776189677.244975:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8a51369 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664388544:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189677.244989:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776189677.244994:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d2b21909 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664388608:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189677.245017:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189677.245067:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664388352 00000100:00100000:1.0:1776189677.245090:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664388608 00000100:00100000:1.0:1776189677.245108:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7022 00000100:00100000:1.0:1776189677.245112:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d82e179d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664388352:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189677.245127:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189677.245164:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d82e179d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664388352:12345-0@lo:13:osp-pre-0-1.0 Request processed in 51us (419us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189677.245188:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7022 00000100:00100000:1.0:1776189677.245201:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7023 00000100:00100000:1.0:1776189677.245205:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000df4082ad pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664388608:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189677.245219:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189677.245245:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000df4082ad pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664388608:12345-0@lo:13:osp-pre-1-1.0 Request processed in 40us (228us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189677.245251:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7023 00000100:00100000:1.0:1776189677.245278:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d2b21909 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664388608:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189677.245298:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664388288 00000100:00100000:1.0:1776189677.245320:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1569 00000100:00100000:1.0:1776189677.245329:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000439c1468 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664388288:12345-0@lo:400:kworker/u8:1.0 00000100:00080000:2.0:1776189677.246002:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189677.246008:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189677.246012:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189677.246018:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9819 00000100:00080000:2.0:1776189677.246021:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00080000:2.0:1776189677.246025:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189677.246047:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000699c9233 x1862459664388672/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189677.246063:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189677.246067:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189677.246068:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1776189677.246072:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bc501d10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664388416:0@lo:400:kworker/u8:1.0 00000100:00080000:2.0:1776189677.246072:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9819 00000100:00080000:2.0:1776189677.246076:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 6s 00000100:00080000:2.0:1776189677.246080:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189677.246092:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002046cfde x1862459664388736/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189677.246095:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3e4a548 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664388352:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189677.246124:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000528f790 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664388800:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189677.246154:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cfec04c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664388864:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189677.246161:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189677.246208:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189677.246221:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cf6eb6bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664388928:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189677.246257:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664388800 00000100:00100000:2.0:1776189677.246288:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664388864 00000100:00100000:2.0:1776189677.246306:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8727 00000100:00100000:2.0:1776189677.246310:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000064636c08 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664388800:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189677.246345:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000064636c08 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664388800:12345-0@lo:400:kworker/u8:1.0 Request processed in 35us (186us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189677.246351:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8727 00000100:00100000:2.0:1776189677.246363:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8728 00000100:00100000:2.0:1776189677.246366:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000aa8506bc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664388864:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189677.246394:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000aa8506bc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664388864:12345-0@lo:400:kworker/u8:1.0 Request processed in 27us (187us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189677.246399:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8728 00000100:00100000:0.0:1776189677.246561:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189677.246619:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cfec04c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664388864:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189677.246641:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000528f790 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664388800:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189677.246657:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664388928 00000100:00100000:0.0:1776189677.246672:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10183 00000100:00100000:0.0:1776189677.246678:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b64538ad pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664388928:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189677.246699:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d3fa5d0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664388480:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189677.246722:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b64538ad pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664388928:12345-0@lo:400:kworker/u8:1.0 Request processed in 42us (162us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189677.246730:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189677.246732:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10183 00000100:00100000:0.0:1776189677.246749:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664388480 00000100:00100000:2.0:1776189677.246751:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000699c9233 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664388672:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189677.246771:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57539 00000100:00100000:0.0:1776189677.246778:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000528f790 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664388480:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189677.246779:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:02000000:0.0:1776189677.246795:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189677.246809:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002046cfde pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664388736:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189677.246844:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189677.246847:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000528f790 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664388480:12345-0@lo:38:kworker/u8:1.0 Request processed in 70us (118us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189677.246855:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57539 00000100:00100000:0.0:1776189677.246869:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664388672 00000100:00100000:0.0:1776189677.246890:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57540 00000100:00100000:0.0:1776189677.246893:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cfec04c5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664388672:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189677.246900:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189677.246923:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cfec04c5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664388672:12345-0@lo:38:kworker/u8:1.0 Request processed in 29us (149us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189677.246929:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57540 00000100:00100000:0.0:1776189677.246951:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cf6eb6bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664388928:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189677.246994:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664388736 00000100:00100000:0.0:1776189677.247012:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9179 00000100:00100000:0.0:1776189677.247015:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000085a66d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664388736:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189677.247022:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189677.247045:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000085a66d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664388736:12345-0@lo:38:kworker/u8:1.0 Request processed in 29us (202us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189677.247051:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9179 00000100:00080000:2.0:1776189677.247210:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189677.247215:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189677.247218:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d3fa5d0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664388480:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776189677.247229:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189677.247231:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189677.247233:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000699c9233 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664388672:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776189677.247240:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189677.247242:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189677.247244:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002046cfde pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664388736:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189677.247359:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000439c1468 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664388288:12345-0@lo:400:kworker/u8:1.0 Request processed in 2028us (2896us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189677.247367:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1569 00000100:00100000:3.0:1776189677.247590:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b9d589b6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664388288:0@lo:400:kworker/u8:1.0 00010000:02020000:0.0:1776189677.300287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189677.557383:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189677.564401:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189677.812378:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189678.068330:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189678.324289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189678.580282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189678.591222:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189678.836354:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189679.095150:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189679.348821:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189679.604290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189679.613536:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189679.860810:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189680.116403:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189680.372318:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189680.628281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189680.636496:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189680.884349:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189681.022782:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189681.022827:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189681.022863:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189681.022954:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654964672 00000100:00100000:3.0:1776189681.023020:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654964736 00000100:00100000:3.0:1776189681.023041:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10184 00000100:00100000:3.0:1776189681.023047:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002046cfde pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654964672:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189681.023122:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002046cfde pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654964672:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 74us (348us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189681.023143:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10184 00000100:00100000:3.0:1776189681.023160:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57541 00000100:00100000:3.0:1776189681.023164:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000699c9233 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654964736:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:3.0:1776189681.023222:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776189681.023248:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000699c9233 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654964736:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 83us (385us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189681.023254:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57541 00000100:00100000:1.0:1776189681.023729:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654964608 00000100:00100000:1.0:1776189681.023771:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1570 00000100:00100000:1.0:1776189681.023776:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006339443e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654964608:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189681.023816:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006339443e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654964608:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 39us (988us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189681.023822:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1570 00010000:02020000:0.0:1776189681.140636:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189681.340574:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000006dfa37f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664388992:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189681.340644:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189681.340703:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664388992 00000100:00100000:2.0:1776189681.340770:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7024 00000100:00100000:2.0:1776189681.340778:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fd041bfd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664388992:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189681.340796:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776189681.340808:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b15f52c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664389056:0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189681.340811:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189681.340845:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189681.340860:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fd041bfd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664388992:12345-0@lo:13:osp-pre-1-0.0 Request processed in 82us (215us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189681.340868:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7024 00000100:00100000:0.0:1776189681.340879:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664389056 00000100:00100000:0.0:1776189681.340920:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7025 00000100:00100000:0.0:1776189681.340925:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c0bb1b99 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664389056:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776189681.340941:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189681.340972:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c0bb1b99 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664389056:12345-0@lo:13:osp-pre-0-0.0 Request processed in 47us (129us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189681.340978:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7025 00000100:00100000:0.0:1776189681.341016:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b15f52c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664389056:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189681.341098:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000006dfa37f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664388992:0@lo:13:osp-pre-1-0.0 00010000:02020000:0.0:1776189681.396582:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189681.653723:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189681.660618:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189681.908338:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189682.165480:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189682.364741:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000db568034 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664389184:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189682.364814:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189682.364923:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776189682.364934:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b15f52c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664389120:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189682.364965:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189682.365061:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664389184 00000100:00100000:0.0:1776189682.365110:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664389120 00000100:00100000:0.0:1776189682.365136:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7026 00000100:00100000:0.0:1776189682.365141:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b7fee202 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664389184:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776189682.365164:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189682.365220:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b7fee202 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664389184:12345-0@lo:13:osp-pre-0-1.0 Request processed in 78us (406us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189682.365228:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7026 00000100:00100000:0.0:1776189682.365246:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7027 00000100:00100000:0.0:1776189682.365250:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e67b9083 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664389120:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189682.365260:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189682.365286:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e67b9083 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664389120:12345-0@lo:13:osp-pre-1-1.0 Request processed in 36us (321us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189682.365292:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7027 00000100:00100000:0.0:1776189682.365335:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000db568034 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664389184:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189682.365600:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b15f52c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664389120:0@lo:13:osp-pre-1-1.0 00000100:00080000:3.0:1776189682.365790:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189682.365794:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189682.365797:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189682.365802:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9824 00000100:00080000:3.0:1776189682.365805:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00080000:3.0:1776189682.365808:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189682.365827:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000018a48c26 x1862459664389376/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189682.365848:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189682.365851:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189682.365853:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189682.365856:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9824 00000100:00080000:3.0:1776189682.365858:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:3.0:1776189682.365861:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189682.365868:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006d50a26f x1862459664389504/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189682.365875:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189682.365878:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189682.365879:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189682.365881:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9824 00000100:00080000:3.0:1776189682.365883:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 11s 00000100:00080000:3.0:1776189682.365885:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189682.365894:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001abd0c8c x1862459664389568/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189682.365942:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000051f898a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664389440:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189682.365974:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189682.365994:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1776189682.365999:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c8af4ac5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664389312:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189682.366021:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189682.366036:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a13b6a88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664389760:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189682.366055:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189682.366081:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664389440 00000100:00100000:3.0:1776189682.366113:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664389312 00000100:00100000:3.0:1776189682.366129:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664389760 00000100:00100000:3.0:1776189682.366144:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10185 00000100:00100000:3.0:1776189682.366149:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d629a97e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664389440:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189682.366215:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d629a97e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664389440:12345-0@lo:400:kworker/u8:0.0 Request processed in 65us (241us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189682.366221:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10185 00000100:00100000:3.0:1776189682.366235:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10186 00000100:00100000:3.0:1776189682.366238:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000ea648da pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664389312:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189682.366264:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000ea648da pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664389312:12345-0@lo:400:kworker/u8:0.0 Request processed in 25us (242us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189682.366269:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10186 00000100:00100000:3.0:1776189682.366281:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10187 00000100:00100000:3.0:1776189682.366284:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000d73684e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664389760:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189682.366307:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000d73684e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664389760:12345-0@lo:400:kworker/u8:0.0 Request processed in 22us (252us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189682.366311:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10187 00000100:00100000:3.0:1776189682.366345:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c8af4ac5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664389312:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189682.366546:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018a48c26 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664389376:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189682.366576:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189682.366593:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d50a26f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664389504:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189682.366613:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189682.366626:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001abd0c8c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664389568:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189682.366648:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189682.366676:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664389376 00000100:00100000:2.0:1776189682.366706:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664389504 00000100:00100000:2.0:1776189682.366716:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57542 00000100:00100000:2.0:1776189682.366719:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d3fa5d0f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664389376:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189682.366732:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189682.366770:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d3fa5d0f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664389376:12345-0@lo:38:kworker/u8:0.0 Request processed in 50us (194us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189682.366775:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57542 00000100:00100000:2.0:1776189682.366786:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57543 00000100:00100000:2.0:1776189682.366789:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000b321ca3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664389504:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189682.366795:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189682.366816:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000b321ca3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664389504:12345-0@lo:38:kworker/u8:0.0 Request processed in 26us (203us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189682.366821:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57543 00000100:00100000:2.0:1776189682.366840:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664389568 00000100:00100000:2.0:1776189682.366858:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9180 00000100:00100000:2.0:1776189682.366861:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001c8daed5 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664389568:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189682.366868:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189682.366892:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001c8daed5 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664389568:12345-0@lo:38:kworker/u8:0.0 Request processed in 30us (245us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189682.366903:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9180 00000100:00080000:2.0:1776189682.366939:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189682.366943:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189682.366947:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018a48c26 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664389376:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776189682.366955:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189682.366957:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189682.366960:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d50a26f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664389504:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776189682.366966:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189682.366968:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189682.366971:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001abd0c8c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664389568:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189682.367059:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000062f2e99c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664389632:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189682.367091:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189682.367121:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1776189682.367129:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000031a76a58 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664389248:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189682.367169:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189682.367202:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eaae2e6e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664389696:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189682.367223:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189682.367244:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664389248 00000100:00100000:1.0:1776189682.367272:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1571 00000100:00100000:1.0:1776189682.367277:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a3281d23 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664389248:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189682.367310:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a3281d23 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664389248:12345-0@lo:400:kworker/u8:0.0 Request processed in 32us (141us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189682.367316:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1571 00000100:00100000:1.0:1776189682.367361:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000031a76a58 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664389248:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189682.367739:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a13b6a88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664389760:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189682.367755:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000051f898a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664389440:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189682.368151:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664389632 00000100:00100000:2.0:1776189682.368221:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664389696 00000100:00100000:2.0:1776189682.368240:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8729 00000100:00100000:2.0:1776189682.368244:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000002c44196 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664389632:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189682.368295:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000002c44196 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664389632:12345-0@lo:400:kworker/u8:0.0 Request processed in 50us (1204us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189682.368302:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8729 00000100:00100000:2.0:1776189682.368315:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8730 00000100:00100000:2.0:1776189682.368318:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000db568034 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664389696:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189682.368345:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000db568034 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664389696:12345-0@lo:400:kworker/u8:0.0 Request processed in 26us (1122us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189682.368358:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8730 00000100:00100000:1.0:1776189682.368570:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eaae2e6e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664389696:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189682.368581:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000062f2e99c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664389632:0@lo:400:kworker/u8:0.0 00010000:02020000:0.0:1776189682.420587:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189682.676290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189682.684491:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189682.937800:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189683.189033:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189683.444291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189683.700257:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189683.708476:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189683.956282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189684.212872:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189684.469101:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189684.724286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189684.734840:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189684.980286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189685.236292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189685.492290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189685.750928:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189685.756640:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189686.004387:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189686.139728:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189686.139789:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189686.139829:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189686.139929:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654964864 00000100:00100000:1.0:1776189686.140049:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654964928 00000100:00100000:1.0:1776189686.140072:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10188 00000100:00100000:1.0:1776189686.140078:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000018d4eed1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654964864:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189686.140213:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654964800 00000100:00100000:3.0:1776189686.140243:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1572 00000100:00100000:3.0:1776189686.140247:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b7303d3d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654964800:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189686.140311:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b7303d3d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654964800:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 63us (522us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189686.140318:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1572 00000100:00100000:1.0:1776189686.140684:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000018d4eed1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654964864:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 604us (964us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189686.140692:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10188 00000100:00100000:1.0:1776189686.140710:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57544 00000100:00100000:1.0:1776189686.140714:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b9d589b6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654964928:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776189686.140728:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189686.140754:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b9d589b6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654964928:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 39us (925us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189686.140761:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57544 00010000:02020000:0.0:1776189686.260491:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189686.462099:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776189686.462123:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c328f1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664389824:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189686.462237:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189686.462699:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000270ba913 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664389888:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189686.462734:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189686.471061:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664389824 00000100:00100000:3.0:1776189686.471137:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664389888 00000100:00100000:3.0:1776189686.471189:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7028 00000100:00100000:3.0:1776189686.471198:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cf6eb6bb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664389824:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189686.471230:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189686.471329:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cf6eb6bb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664389824:12345-0@lo:13:osp-pre-0-0.0 Request processed in 132us (9093us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189686.471337:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7028 00000100:00100000:3.0:1776189686.471355:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7029 00000100:00100000:3.0:1776189686.471359:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000266f5a0c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664389888:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776189686.471376:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189686.471405:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000266f5a0c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664389888:12345-0@lo:13:osp-pre-1-0.0 Request processed in 45us (8672us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189686.471410:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7029 00000100:00100000:3.0:1776189686.471475:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c328f1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664389824:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189686.471720:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000270ba913 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664389888:0@lo:13:osp-pre-1-0.0 00010000:02020000:0.0:1776189686.518550:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189686.772606:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189686.783719:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189687.028359:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189687.284340:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:3.0:1776189687.484375:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189687.484381:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1776189687.484384:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001e60ceac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664389952:0@lo:13:osp-pre-1-1.0 00000100:00080000:3.0:1776189687.484384:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189687.484394:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9829 00000100:00080000:3.0:1776189687.484398:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1776189687.484403:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189687.484428:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c8af4ac5 x1862459664390144/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189687.484454:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1776189687.484458:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1776189687.484471:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189687.484473:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189687.484477:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9829 00000100:00080000:3.0:1776189687.484480:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1776189687.484484:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189687.484497:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776189687.484497:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000062f2e99c x1862459664390272/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189687.484506:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000008d7a449 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664390208:0@lo:400:kworker/u8:0.0 00000100:00080000:3.0:1776189687.484508:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189687.484513:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189687.484514:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189687.484517:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9829 00000100:00080000:3.0:1776189687.484519:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1776189687.484523:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189687.484536:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000eaae2e6e x1862459664390336/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189687.484542:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189687.484567:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000031a76a58 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664390400:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189687.484617:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000051f898a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664390016:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189687.484618:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189687.484639:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776189687.484645:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018a48c26 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664390592:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189687.484732:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776189687.484740:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088ec3431 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664390656:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189687.484818:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664389952 00000100:00100000:2.0:1776189687.484861:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7030 00000100:00100000:2.0:1776189687.484867:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000a0e79be pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664389952:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189687.484888:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189687.485851:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189687.486044:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664390400 00000100:00100000:0.0:1776189687.486080:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664390592 00000100:00100000:0.0:1776189687.486099:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8731 00000100:00100000:0.0:1776189687.486104:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fc1e1957 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664390400:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189687.486162:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fc1e1957 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664390400:12345-0@lo:400:kworker/u8:0.0 Request processed in 57us (1547us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189687.486169:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8731 00000100:00100000:0.0:1776189687.486199:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8732 00000100:00100000:0.0:1776189687.486203:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000087a87f6a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664390592:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189687.486230:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000087a87f6a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664390592:12345-0@lo:400:kworker/u8:0.0 Request processed in 26us (384us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189687.486235:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8732 00000100:00100000:0.0:1776189687.486262:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664390208 00000100:00100000:0.0:1776189687.486286:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10189 00000100:00100000:0.0:1776189687.486289:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c4620d7d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664390208:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189687.486857:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c4620d7d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664390208:12345-0@lo:400:kworker/u8:0.0 Request processed in 563us (2316us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189687.486983:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10189 00000100:00100000:0.0:1776189687.487034:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018a48c26 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664390592:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189687.487045:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000031a76a58 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664390400:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189687.487054:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000008d7a449 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664390208:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189687.487117:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a13b6a88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664390080:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189687.487150:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189687.487167:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d50a26f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664390528:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189687.488163:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189687.488216:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001abd0c8c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664390464:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189687.488259:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189687.488303:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c8af4ac5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664390144:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189687.488329:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189687.488345:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000062f2e99c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664390272:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189687.488366:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189687.488380:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eaae2e6e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664390336:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189687.488405:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189687.488438:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664390464 00000100:00100000:3.0:1776189687.488485:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8733 00000100:00100000:3.0:1776189687.488490:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000024796ea pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664390464:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189687.488538:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000024796ea pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664390464:12345-0@lo:400:kworker/u8:0.0 Request processed in 48us (279us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189687.488545:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8733 00000100:00100000:3.0:1776189687.489077:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664390016 00000100:00100000:3.0:1776189687.489113:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1573 00000100:00100000:3.0:1776189687.489117:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000089048673 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664390016:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189687.489154:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000089048673 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664390016:12345-0@lo:400:kworker/u8:0.0 Request processed in 36us (997us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189687.489161:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1573 00000100:00100000:3.0:1776189687.489233:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000051f898a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664390016:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189687.489246:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001abd0c8c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664390464:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189687.489291:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664390336 00000100:00100000:3.0:1776189687.489310:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9181 00000100:00100000:3.0:1776189687.489313:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000031a76a58 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664390336:12345-0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189687.489317:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:02000000:3.0:1776189687.489327:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189687.489351:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664390080 00000100:00100000:3.0:1776189687.489358:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000031a76a58 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664390336:12345-0@lo:38:kworker/u8:0.0 Request processed in 44us (953us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189687.489364:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9181 00000100:00100000:0.0:1776189687.489374:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664390144 00000100:00080000:3.0:1776189687.489386:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189687.489390:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189687.489393:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eaae2e6e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664390336:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189687.489408:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664390272 00000100:00100000:3.0:1776189687.489416:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664390528 00000100:00100000:0.0:1776189687.489420:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10190 00000100:00100000:0.0:1776189687.489424:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003987cbc7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664390080:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189687.489429:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10191 00000100:00100000:3.0:1776189687.489432:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c875033e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664390528:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189687.489537:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c875033e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664390528:12345-0@lo:400:kworker/u8:0.0 Request processed in 103us (220us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189687.489544:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10191 00000100:00100000:3.0:1776189687.489559:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57545 00000100:00100000:3.0:1776189687.489562:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002126a7bf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664390144:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189687.489817:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189687.491456:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189687.491536:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57546 00000100:00100000:1.0:1776189687.491546:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000008d7a449 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664390272:12345-0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189687.491559:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002126a7bf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664390144:12345-0@lo:38:kworker/u8:0.0 Request processed in 1994us (3227us total) trans 0 rc -19/-19 00010000:02000000:1.0:1776189687.491564:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189687.491608:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000008d7a449 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664390272:12345-0@lo:38:kworker/u8:0.0 Request processed in 62us (3242us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189687.491615:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57546 00000100:00100000:1.0:1776189687.491650:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664390656 00000100:00100000:1.0:1776189687.491684:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7031 00000100:00100000:1.0:1776189687.491687:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004920d098 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664390656:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189687.491708:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189687.491781:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57545 00000100:00100000:0.0:1776189687.491824:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003987cbc7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664390080:12345-0@lo:400:kworker/u8:0.0 Request processed in 2397us (4673us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189687.491832:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10190 00000100:00100000:2.0:1776189687.491847:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000a0e79be pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664389952:12345-0@lo:13:osp-pre-1-1.0 Request processed in 6977us (7389us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189687.491855:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7030 00000100:00100000:0.0:1776189687.491896:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a13b6a88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664390080:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189687.491911:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d50a26f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664390528:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189687.492220:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001e60ceac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664389952:0@lo:13:osp-pre-1-1.0 00000100:00080000:3.0:1776189687.492376:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189687.492381:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189687.492386:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c8af4ac5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664390144:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776189687.492399:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189687.492402:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189687.492404:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000062f2e99c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664390272:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189687.499467:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004920d098 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664390656:12345-0@lo:13:osp-pre-0-1.0 Request processed in 7770us (8010us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189687.499483:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7031 00000100:00100000:0.0:1776189687.499674:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088ec3431 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664390656:0@lo:13:osp-pre-0-1.0 00010000:02020000:0.0:1776189687.541369:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189687.797414:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189687.804390:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189688.053050:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189688.308325:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189688.564306:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189688.820326:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189688.828506:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189689.077139:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189689.334505:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189689.588299:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189689.846241:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189689.852372:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189690.100353:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189690.356306:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189690.612286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189690.868318:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:1.0:1776189690.876702:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189691.124273:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189691.259146:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189691.259449:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189691.259506:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189691.259564:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654964992 00000100:00100000:1.0:1776189691.259624:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1574 00000100:00100000:1.0:1776189691.259631:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000007949e4c pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654964992:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189691.259700:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000007949e4c pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654964992:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 68us (252us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189691.259707:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1574 00000100:00100000:0.0:1776189691.265406:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654965056 00000100:00100000:0.0:1776189691.265485:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10192 00000100:00100000:0.0:1776189691.265824:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000062f2e99c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654965056:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189691.265989:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000062f2e99c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654965056:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 166us (6850us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189691.265998:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10192 00000100:00100000:1.0:1776189691.267834:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654965120 00000100:00100000:1.0:1776189691.267874:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57547 00000100:00100000:1.0:1776189691.267880:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c8af4ac5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654965120:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776189691.267897:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189691.267941:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c8af4ac5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2277:x1862459654965120:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 61us (8437us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189691.267947:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57547 00010000:02020000:0.0:1776189691.384070:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189691.580951:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eaae2e6e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664390720:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189691.580991:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094837a0a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664390784:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189691.581030:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189691.581075:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664390720 00000100:00100000:2.0:1776189691.581147:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7032 00000100:00100000:2.0:1776189691.581156:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ace99b9b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664390720:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189691.581230:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189691.581990:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ace99b9b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664390720:12345-0@lo:13:osp-pre-0-0.0 Request processed in 829us (958us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189691.582003:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eaae2e6e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664390720:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189691.582005:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7032 00000100:00100000:0.0:1776189691.582079:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189691.582148:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664390784 00000100:00100000:2.0:1776189691.582199:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7033 00000100:00100000:2.0:1776189691.582203:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001e60ceac pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664390784:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189691.582223:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189691.583448:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001e60ceac pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664390784:12345-0@lo:13:osp-pre-1-0.0 Request processed in 1229us (1355us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189691.583460:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7033 00000100:00100000:1.0:1776189691.585501:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094837a0a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664390784:0@lo:13:osp-pre-1-0.0 00010000:02020000:0.0:1776189691.636296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189691.892884:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:3.0:1776189691.900316:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189692.148291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189692.404438:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189692.604458:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088ec3431 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664390848:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189692.604539:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1776189692.604639:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189692.604645:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189692.604648:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189692.604653:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9834 00000100:00080000:3.0:1776189692.604657:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189692.604662:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189692.604675:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664390848 00000100:00100000:3.0:1776189692.604682:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000018a48c26 x1862459664391040/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189692.604711:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189692.604714:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189692.604715:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189692.604719:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9834 00000100:00080000:3.0:1776189692.604721:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1776189692.604725:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7034 00000100:00080000:3.0:1776189692.604726:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189692.604733:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000270ba913 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664390848:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189692.604740:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006d50a26f x1862459664391168/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189692.604750:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00002000:00000024:1.0:1776189692.604755:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189692.604850:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001abd0c8c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664390912:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189692.605276:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000051f898a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664390976:0@lo:400:kworker/u8:1.0 00000100:00080000:3.0:1776189692.605959:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189692.605961:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189692.605966:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9834 00000100:00080000:3.0:1776189692.605969:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189692.605973:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189692.606409:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a13b6a88 x1862459664391232/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189692.606605:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018a48c26 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664391040:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189692.610102:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189692.610149:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d50a26f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664391168:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189692.610440:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000270ba913 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664390848:12345-0@lo:13:osp-pre-0-1.0 Request processed in 5705us (5898us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189692.610452:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7034 00000100:00100000:1.0:1776189692.610510:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eaae2e6e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664391104:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189692.611043:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189692.611069:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a13b6a88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664391232:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189692.611105:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189692.611151:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664391232 00000100:00100000:2.0:1776189692.611408:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189692.611439:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000013c0eaff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664391360:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189692.611477:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189692.611499:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776189692.611505:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ec15c7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664391296:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189692.611514:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189692.611536:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189692.611551:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c094ed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664391424:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189692.611594:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189692.611619:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664390912 00000100:00100000:0.0:1776189692.611635:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664391040 00000100:00100000:2.0:1776189692.611651:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1575 00000100:00100000:2.0:1776189692.611657:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c1af929a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664390912:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189692.611701:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c1af929a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664390912:12345-0@lo:400:kworker/u8:1.0 Request processed in 44us (359us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189692.611708:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1575 00000100:00100000:2.0:1776189692.611737:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664391360 00000100:00100000:2.0:1776189692.611757:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664391296 00000100:00100000:2.0:1776189692.611773:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8734 00000100:00100000:2.0:1776189692.611776:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bc501d10 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664391360:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189692.611805:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bc501d10 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664391360:12345-0@lo:400:kworker/u8:1.0 Request processed in 28us (338us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189692.611811:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8734 00000100:00100000:2.0:1776189692.611821:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8735 00000100:00100000:2.0:1776189692.611824:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b8a51369 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664391296:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189692.611847:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b8a51369 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664391296:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (314us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189692.611852:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8735 00000100:00100000:2.0:1776189692.611886:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ec15c7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664391296:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189692.611911:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000013c0eaff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664391360:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189692.611920:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001abd0c8c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664390912:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189692.611942:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776189692.611947:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a7368321 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664391552:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189692.611970:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189692.611993:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664391552 00000100:00100000:2.0:1776189692.612015:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7035 00000100:00100000:2.0:1776189692.612018:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053d4c624 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664391552:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189692.612030:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189692.612061:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053d4c624 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664391552:12345-0@lo:13:osp-pre-1-1.0 Request processed in 41us (90us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189692.612067:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7035 00000100:00100000:2.0:1776189692.612092:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a7368321 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664391552:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189692.612137:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088ec3431 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664390848:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189692.612187:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189692.612208:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000036a45663 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664391488:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189692.612215:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9182 00000100:00100000:3.0:1776189692.612223:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000071c1b92f pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664391232:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776189692.612237:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189692.612276:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000071c1b92f pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664391232:12345-0@lo:38:kworker/u8:1.0 Request processed in 54us (1170us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189692.612498:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664391168 00000100:00100000:0.0:1776189692.612511:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664390976 00000100:00100000:0.0:1776189692.612525:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664391424 00000100:00100000:0.0:1776189692.612534:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664391104 00000100:00100000:0.0:1776189692.612543:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10193 00000100:00100000:0.0:1776189692.612546:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001c328f1b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664390976:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189692.612568:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189692.612594:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001c328f1b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3361:x1862459664390976:12345-0@lo:400:kworker/u8:1.0 Request processed in 45us (1081us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189692.612602:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10193 00000100:00100000:1.0:1776189692.612611:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10194 00000100:00100000:1.0:1776189692.612617:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b15f52c9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664391424:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189692.612620:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10195 00000100:00100000:0.0:1776189692.612627:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000088ec3431 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664391104:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189692.612656:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b15f52c9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664391424:12345-0@lo:400:kworker/u8:1.0 Request processed in 39us (1064us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189692.612663:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10194 00000100:00100000:1.0:1776189692.612676:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57548 00000100:00100000:1.0:1776189692.612679:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0b45fb4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664391040:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189692.612688:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189692.612710:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0b45fb4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664391040:12345-0@lo:38:kworker/u8:1.0 Request processed in 29us (2616us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189692.612715:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57548 00000100:00100000:1.0:1776189692.612726:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57549 00000100:00100000:1.0:1776189692.612728:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001ce461df pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664391168:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189692.612734:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189692.612753:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001ce461df pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664391168:12345-0@lo:38:kworker/u8:1.0 Request processed in 24us (1713us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189692.612758:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57549 00000100:00100000:1.0:1776189692.612775:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664391488 00000100:00100000:1.0:1776189692.612795:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8736 00000100:00100000:1.0:1776189692.612798:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000797ba66f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664391488:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189692.612824:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000797ba66f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664391488:12345-0@lo:400:kworker/u8:1.0 Request processed in 25us (258us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189692.612829:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8736 00000100:00100000:1.0:1776189692.612867:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000036a45663 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664391488:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189692.612876:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000088ec3431 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664391104:12345-0@lo:400:kworker/u8:1.0 Request processed in 247us (702us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189692.612884:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10195 00000100:00100000:1.0:1776189692.612887:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eaae2e6e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664391104:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189692.613191:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c094ed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664391424:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189692.613205:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000051f898a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3361:1862459664390976:0@lo:400:kworker/u8:1.0 00000100:00080000:0.0:1776189692.613252:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189692.613256:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189692.613260:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018a48c26 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664391040:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776189692.613267:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189692.613270:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189692.613272:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d50a26f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664391168:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776189692.613281:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189692.613284:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189692.613285:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a13b6a88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664391232:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189692.614288:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9182 00010000:02020000:0.0:1776189692.660307:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189692.916489:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189692.924414:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189693.172308:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189693.432810:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189693.684327:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189693.940386:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189693.948556:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189694.196457:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189694.456975:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189694.708284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189694.964355:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189694.972380:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189695.220407:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189695.476280:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189695.732290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189695.988286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189695.997054:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189696.244603:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189696.379418:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189696.379459:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189696.379534:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189696.379600:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654965248 00000100:00100000:0.0:1776189696.379660:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654965184 00000100:00100000:0.0:1776189696.379713:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1576 00000100:00100000:0.0:1776189696.379720:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e2219b01 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654965184:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1776189696.379783:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e2219b01 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654965184:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 63us (324us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189696.379790:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1576 00000100:00100000:0.0:1776189696.379823:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654965312 00000100:00100000:0.0:1776189696.379857:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57550 00000100:00100000:0.0:1776189696.379861:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000036a45663 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654965312:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:0.0:1776189696.379875:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189696.379900:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000036a45663 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654965312:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 39us (367us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189696.379905:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57550 00000100:00100000:3.0:1776189696.380013:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10196 00000100:00100000:3.0:1776189696.380018:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000073262ad2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654965248:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1776189696.380048:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000073262ad2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654965248:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 30us (639us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189696.380054:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10196 00010000:02020000:2.0:1776189696.500303:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189696.700869:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009257ba59 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664391616:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189696.700934:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189696.701048:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664391616 00000100:00100000:3.0:1776189696.701096:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7036 00000100:00100000:3.0:1776189696.701102:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a7368321 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664391616:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189696.701130:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189696.701206:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a7368321 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664391616:12345-0@lo:13:osp-pre-0-0.0 Request processed in 104us (272us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189696.701214:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7036 00000100:00100000:1.0:1776189696.701339:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009257ba59 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664391616:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189696.702209:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000006dfa37f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664391680:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189696.702248:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189696.702871:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664391680 00000100:00100000:3.0:1776189696.702916:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7037 00000100:00100000:3.0:1776189696.702920:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bb631996 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664391680:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776189696.702944:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189696.702994:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bb631996 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664391680:12345-0@lo:13:osp-pre-1-0.0 Request processed in 73us (746us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189696.703001:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7037 00000100:00100000:0.0:1776189696.703363:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000006dfa37f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664391680:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189696.756371:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189697.012231:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189697.020396:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189697.268565:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189697.525066:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:3.0:1776189697.724404:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189697.724412:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189697.724415:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1776189697.724419:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00080000:3.0:1776189697.724423:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9839 00000100:00080000:3.0:1776189697.724425:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189697.724430:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189697.724436:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e5ef59b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664391744:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189697.724453:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009fddac3a x1862459664391872/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189697.724475:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189697.724478:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189697.724479:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189697.724482:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9839 00000100:00080000:3.0:1776189697.724484:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189697.724487:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189697.724496:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e071bb1b x1862459664392000/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189697.724502:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189697.724505:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189697.724506:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1776189697.724509:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1776189697.724509:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9839 00000100:00080000:3.0:1776189697.724511:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776189697.724513:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189697.724522:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009257ba59 x1862459664392064/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189697.724553:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009fddac3a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664391872:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189697.724569:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f61bb34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664391808:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189697.724578:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189697.724592:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e071bb1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664392000:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189697.724612:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189697.724637:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009257ba59 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664392064:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189697.724659:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189697.724694:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664391872 00000100:00100000:0.0:1776189697.724722:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664392000 00000100:00100000:0.0:1776189697.724738:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57551 00000100:00100000:0.0:1776189697.724755:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000006dfa37f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664391872:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189697.724772:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189697.724772:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000058c450e1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664392192:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189697.724802:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189697.724808:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000006dfa37f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664391872:12345-0@lo:38:kworker/u8:0.0 Request processed in 54us (230us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189697.724814:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57551 00000100:00100000:1.0:1776189697.724820:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1776189697.724824:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3185a52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664392128:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189697.724828:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57552 00000100:00100000:0.0:1776189697.724831:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007a0ce2a1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664392000:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189697.724837:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189697.724844:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189697.724855:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007a0ce2a1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664392000:12345-0@lo:38:kworker/u8:0.0 Request processed in 23us (243us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189697.724856:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000048c7c2e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664392384:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189697.724860:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57552 00000100:00100000:1.0:1776189697.724874:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189697.724878:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664392064 00000100:00100000:0.0:1776189697.724893:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9183 00000100:00100000:0.0:1776189697.724896:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000067bb60df pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664392064:12345-0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189697.724899:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664392192 00010000:02000000:0.0:1776189697.724902:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189697.724922:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000067bb60df pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664392064:12345-0@lo:38:kworker/u8:0.0 Request processed in 25us (263us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189697.724924:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664392128 00000100:00100000:0.0:1776189697.724928:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9183 00000100:00100000:1.0:1776189697.724950:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664392384 00000100:00100000:0.0:1776189697.724958:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664391744 00000100:00100000:1.0:1776189697.724965:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8737 00000100:00100000:0.0:1776189697.724978:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1577 00000100:00100000:0.0:1776189697.724981:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003e184549 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664391744:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189697.725013:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003e184549 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664391744:12345-0@lo:400:kworker/u8:0.0 Request processed in 30us (505us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189697.725018:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1577 00000100:00080000:0.0:1776189697.725062:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189697.725066:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189697.725069:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009fddac3a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664391872:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776189697.725078:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189697.725080:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189697.725082:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e071bb1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664392000:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776189697.725089:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189697.725090:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189697.725092:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009257ba59 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664392064:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189697.725122:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e5ef59b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664391744:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189697.725242:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189697.725259:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f34aeb18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664392256:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189697.725646:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000114609f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664392192:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189697.725647:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8738 00000100:00100000:0.0:1776189697.725651:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000f2a9137 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664392128:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189697.725712:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a388dd2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664392448:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189697.725746:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189697.728354:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189697.728447:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000080b4f511 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664392512:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189697.728490:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189697.728522:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1776189697.728529:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c6f4c7a3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664391936:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189697.728562:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189697.728581:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005ee0b9e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664392320:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189697.728606:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189697.728652:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664392320 00000100:00100000:3.0:1776189697.728700:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8739 00000100:00100000:3.0:1776189697.728706:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b7713fdb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664392384:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189697.730485:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000f2a9137 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664392128:12345-0@lo:400:kworker/u8:0.0 Request processed in 4824us (5631us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189697.730486:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000114609f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664392192:12345-0@lo:400:kworker/u8:0.0 Request processed in 4836us (5680us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189697.730512:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8737 00000100:00100000:1.0:1776189697.730707:0:186995:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8740 00000100:00100000:1.0:1776189697.730715:0:186995:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000099b8eb0d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664392320:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189697.730805:0:186995:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000099b8eb0d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_023:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664392320:12345-0@lo:400:kworker/u8:0.0 Request processed in 87us (2196us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189697.730812:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8738 00000100:00100000:1.0:1776189697.730887:0:186995:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8740 00000100:00100000:0.0:1776189697.730907:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664391808 00000100:00100000:1.0:1776189697.730941:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664392448 00000100:00100000:0.0:1776189697.730947:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664392256 00000100:00100000:0.0:1776189697.730991:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664391936 00000100:00100000:0.0:1776189697.731012:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10197 00000100:00100000:0.0:1776189697.731016:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001a975e34 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664391808:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189697.731052:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001a975e34 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664391808:12345-0@lo:400:kworker/u8:0.0 Request processed in 35us (5810us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189697.731058:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10197 00000100:00100000:0.0:1776189697.731070:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10198 00000100:00100000:0.0:1776189697.731085:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001fe1325f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664392256:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189697.731111:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001fe1325f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664392256:12345-0@lo:400:kworker/u8:0.0 Request processed in 26us (2767us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189697.731116:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10198 00000100:00100000:0.0:1776189697.731140:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10199 00000100:00100000:0.0:1776189697.731143:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000df4082ad pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664391936:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189697.731192:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000df4082ad pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664391936:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (2605us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189697.731198:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10199 00000100:00100000:3.0:1776189697.732935:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b7713fdb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664392384:12345-0@lo:400:kworker/u8:0.0 Request processed in 4223us (8054us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189697.732949:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8739 00000100:00100000:3.0:1776189697.733033:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c6f4c7a3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664391936:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189697.733045:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005ee0b9e5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664392320:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189697.733056:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f61bb34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664391808:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189697.733066:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f34aeb18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664392256:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189697.733138:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664392512 00000100:00100000:2.0:1776189697.733890:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3185a52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664392128:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189697.733906:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000048c7c2e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664392384:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189697.733919:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000058c450e1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664392192:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189697.736716:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7038 00000100:00100000:0.0:1776189697.736731:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006ad1d61a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664392448:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776189697.736765:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189697.736850:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006ad1d61a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664392448:12345-0@lo:13:osp-pre-0-1.0 Request processed in 121us (11103us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189697.736856:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7038 00000100:00100000:0.0:1776189697.736885:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7039 00000100:00100000:0.0:1776189697.736889:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004ac5ba05 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664392512:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189697.736899:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189697.736930:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004ac5ba05 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664392512:12345-0@lo:13:osp-pre-1-1.0 Request processed in 40us (8439us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189697.736935:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7039 00000100:00100000:2.0:1776189697.737456:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000080b4f511 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664392512:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189697.737964:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a388dd2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664392448:0@lo:13:osp-pre-0-1.0 00010000:02020000:2.0:1776189697.780293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189698.036362:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189698.046021:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189698.292895:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189698.548552:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189698.804300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189699.060282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189699.070515:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189699.316336:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189699.573244:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189699.828252:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189700.085133:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189700.093232:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189700.340503:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189700.596853:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189700.852304:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189701.108446:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189701.116418:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189701.365081:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189701.498548:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189701.498679:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654965440 00000100:00100000:0.0:1776189701.498738:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10200 00000100:00100000:0.0:1776189701.498745:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000266f5a0c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654965440:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1776189701.498828:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000266f5a0c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654965440:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 82us (289us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189701.498835:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10200 00000100:00100000:2.0:1776189701.503592:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189701.503648:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189701.503872:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654965504 00000100:00100000:3.0:1776189701.503926:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57553 00000100:00100000:3.0:1776189701.503932:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000080b4f511 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654965504:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:3.0:1776189701.503951:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776189701.504016:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000080b4f511 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654965504:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 82us (367us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189701.504023:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57553 00000100:00100000:0.0:1776189701.504097:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654965376 00000100:00100000:0.0:1776189701.504136:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1578 00000100:00100000:0.0:1776189701.504140:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000089ddaa97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654965376:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1776189701.504207:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000089ddaa97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654965376:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 66us (624us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189701.504214:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1578 00010000:02020000:2.0:1776189701.621914:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189701.820391:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776189701.821030:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064636c08 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664392640:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189701.821112:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189701.821323:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664392640 00000100:00100000:3.0:1776189701.821371:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7040 00000100:00100000:3.0:1776189701.821377:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000308dffba pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664392640:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189701.821406:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189701.821448:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000308dffba pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664392640:12345-0@lo:13:osp-pre-0-0.0 Request processed in 70us (337us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189701.821455:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7040 00000100:00100000:1.0:1776189701.821788:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064636c08 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664392640:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189701.822031:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000070105d1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664392576:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189701.822069:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189701.822351:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664392576 00000100:00100000:1.0:1776189701.822421:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7041 00000100:00100000:1.0:1776189701.822426:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e13527ae pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664392576:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189701.822445:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189701.822485:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e13527ae pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664392576:12345-0@lo:13:osp-pre-1-0.0 Request processed in 59us (416us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189701.822491:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7041 00000100:00100000:2.0:1776189701.822548:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000070105d1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664392576:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189701.876307:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189702.132390:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189702.140515:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189702.388403:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189702.644307:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:1.0:1776189702.844413:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189702.844421:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189702.844425:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189702.844432:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9844 00000100:00100000:2.0:1776189702.844433:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b9ada007 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664392768:0@lo:400:kworker/u8:1.0 00000100:00080000:1.0:1776189702.844436:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776189702.844442:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189702.844462:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a969b6e4 x1862459664392896/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189702.844484:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189702.844487:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189702.844489:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189702.844492:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9844 00000100:00080000:1.0:1776189702.844493:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776189702.844496:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189702.844505:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008a7066b0 x1862459664393024/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189702.844515:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776189702.844516:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189702.844520:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189702.844522:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189702.844525:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9844 00000100:00080000:1.0:1776189702.844527:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776189702.844530:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189702.844544:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c6bc9cdf x1862459664393088/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189702.844550:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776189702.844558:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b5045f9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664392832:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189702.844594:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189702.844603:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d5cfaed8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664393280:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189702.844619:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005cdf5f34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664393216:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189702.844652:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189702.844692:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664392832 00000100:00100000:2.0:1776189702.844732:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10201 00000100:00100000:2.0:1776189702.844737:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008870952c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664392832:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189702.844787:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008870952c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664392832:12345-0@lo:400:kworker/u8:1.0 Request processed in 48us (191us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189702.844793:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10201 00000100:00100000:3.0:1776189702.844815:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000761163f9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664392704:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189702.844832:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664393216 00000100:00100000:3.0:1776189702.844850:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189702.844863:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8741 00000100:00100000:2.0:1776189702.844868:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002ea4a019 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664393216:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189702.844872:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d52de5d9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664393152:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189702.844908:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189702.844914:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002ea4a019 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664393216:12345-0@lo:400:kworker/u8:1.0 Request processed in 44us (262us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189702.844921:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8741 00000100:00100000:3.0:1776189702.844930:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d4382e95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664393408:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189702.844935:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664393152 00000100:00100000:3.0:1776189702.844958:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189702.844962:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8742 00000100:00100000:2.0:1776189702.844967:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ace99b9b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664393152:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189702.844977:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1776189702.844985:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000026520930 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664392960:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189702.845043:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189702.845226:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ace99b9b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664393152:12345-0@lo:400:kworker/u8:1.0 Request processed in 157us (218us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189702.845232:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8742 00000100:00100000:2.0:1776189702.845255:0:188934:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664393408 00000100:00100000:2.0:1776189702.845278:0:188934:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8743 00000100:00100000:2.0:1776189702.845282:0:188934:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000048c7c2e3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664393408:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189702.845311:0:188934:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000048c7c2e3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_030:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664393408:12345-0@lo:400:kworker/u8:1.0 Request processed in 29us (355us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189702.845317:0:188934:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8743 00000100:00100000:2.0:1776189702.845340:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664393280 00000100:00100000:2.0:1776189702.845358:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10202 00000100:00100000:2.0:1776189702.845362:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c27afb9e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664393280:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189702.845389:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c27afb9e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664393280:12345-0@lo:400:kworker/u8:1.0 Request processed in 26us (751us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189702.845394:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10202 00000100:00100000:2.0:1776189702.845431:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005cdf5f34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664393216:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189702.845441:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b5045f9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664392832:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189702.845491:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664392768 00000100:00100000:2.0:1776189702.845516:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1579 00000100:00100000:2.0:1776189702.845520:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006e7c9c41 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664392768:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189702.845552:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006e7c9c41 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664392768:12345-0@lo:400:kworker/u8:1.0 Request processed in 30us (1041us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189702.845558:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1579 00000100:00100000:2.0:1776189702.845586:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b9ada007 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664392768:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189702.845612:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664392704 00000100:00100000:2.0:1776189702.845633:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7042 00000100:00100000:2.0:1776189702.845636:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004920d098 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664392704:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189702.845650:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189702.845677:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004920d098 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664392704:12345-0@lo:13:osp-pre-1-1.0 Request processed in 40us (829us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189702.846284:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7042 00000100:00100000:0.0:1776189702.847385:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a969b6e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664392896:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189702.848650:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a02f1c47 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664393472:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189702.848707:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189702.848771:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664393472 00000100:00100000:2.0:1776189702.848867:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7043 00000100:00100000:2.0:1776189702.848873:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d2b21909 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664393472:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189702.848894:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189702.848937:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d2b21909 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664393472:12345-0@lo:13:osp-pre-0-1.0 Request processed in 64us (231us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189702.848944:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7043 00000100:00100000:0.0:1776189702.850376:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a02f1c47 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664393472:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189702.851876:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d5cfaed8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664393280:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189702.851924:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189702.851954:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a7066b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664393024:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189702.851957:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664392896 00000100:00100000:0.0:1776189702.852000:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189702.852004:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57554 00000100:00100000:1.0:1776189702.852011:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ab0cd203 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664392896:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189702.852019:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c6bc9cdf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664393088:0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189702.852027:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189702.852063:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189702.852075:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ab0cd203 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664392896:12345-0@lo:38:kworker/u8:1.0 Request processed in 62us (152us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189702.852082:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57554 00000100:00080000:0.0:1776189702.852096:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:1.0:1776189702.852099:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664393024 00000100:00080000:0.0:1776189702.852101:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189702.852106:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a969b6e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664392896:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189702.852122:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57555 00000100:00100000:1.0:1776189702.852127:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000093b814ce pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664393024:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189702.852134:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189702.852147:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664393088 00000100:00100000:1.0:1776189702.852166:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000093b814ce pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664393024:12345-0@lo:38:kworker/u8:1.0 Request processed in 37us (166us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189702.852193:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57555 00000100:00100000:0.0:1776189702.852198:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9184 00000100:00100000:0.0:1776189702.852203:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000070105d1b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664393088:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189702.852213:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00080000:1.0:1776189702.852225:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189702.852230:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189702.852234:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a7066b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664393024:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189702.852256:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000070105d1b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664393088:12345-0@lo:38:kworker/u8:1.0 Request processed in 41us (184us total) trans 0 rc -19/-19 00000100:00080000:1.0:1776189702.852260:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:0.0:1776189702.852264:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9184 00000100:00080000:1.0:1776189702.852265:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189702.852270:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c6bc9cdf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664393088:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189702.852953:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189702.852986:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007f770c5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664393344:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189702.853001:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664392960 00000100:00100000:3.0:1776189702.853021:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189702.853053:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10203 00000100:00100000:2.0:1776189702.853059:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f34aeb18 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664392960:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189702.853065:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000761163f9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664392704:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189702.853080:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d52de5d9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664393152:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189702.853090:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d4382e95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664393408:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189702.853109:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f34aeb18 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664392960:12345-0@lo:400:kworker/u8:1.0 Request processed in 49us (158us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189702.853117:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10203 00000100:00100000:3.0:1776189702.853127:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000026520930 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664392960:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189702.853290:0:188944:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664393344 00000100:00100000:1.0:1776189702.853411:0:188944:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8744 00000100:00100000:1.0:1776189702.853416:0:188944:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005ee0b9e5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664393344:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189702.853460:0:188944:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005ee0b9e5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_031:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664393344:12345-0@lo:400:kworker/u8:1.0 Request processed in 43us (440us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189702.853466:0:188944:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8744 00000100:00100000:3.0:1776189702.853569:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007f770c5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664393344:0@lo:400:kworker/u8:1.0 00010000:02020000:2.0:1776189702.900354:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189703.156858:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189703.166969:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189703.412287:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189703.668558:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189703.924303:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189704.180296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189704.194629:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189704.436292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189704.692298:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189704.948312:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189705.205826:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189705.212496:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189705.461012:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189705.716338:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189705.973032:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189706.228296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189706.237833:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189706.484292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189706.628930:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189706.629040:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189706.635144:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189706.635502:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654965568 00000100:00100000:3.0:1776189706.635560:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1580 00000100:00100000:3.0:1776189706.635567:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d3d96460 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654965568:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189706.635653:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d3d96460 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654965568:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 86us (517us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189706.635660:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1580 00000100:00100000:2.0:1776189706.639272:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654965696 00000100:00100000:3.0:1776189706.640141:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654965632 00000100:00100000:3.0:1776189706.640236:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10204 00000100:00100000:3.0:1776189706.640243:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000026520930 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654965632:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189706.640320:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000026520930 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654965632:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 76us (11280us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189706.640327:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10204 00000100:00100000:3.0:1776189706.640346:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57556 00000100:00100000:3.0:1776189706.640350:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000068da6036 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654965696:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:3.0:1776189706.640365:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776189706.640386:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000068da6036 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654965696:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 35us (11466us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189706.640392:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57556 00010000:02020000:2.0:1776189706.740311:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189706.940381:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1776189706.940403:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064636c08 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664393600:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189706.940484:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189706.940522:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a7066b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664393536:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189706.940544:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189706.940650:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664393600 00000100:00100000:0.0:1776189706.940696:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664393536 00000100:00100000:0.0:1776189706.940725:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7044 00000100:00100000:0.0:1776189706.940730:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000087b82cd7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664393600:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189706.940769:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189706.940823:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000087b82cd7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664393600:12345-0@lo:13:osp-pre-1-0.0 Request processed in 91us (339us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189706.940831:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7044 00000100:00100000:0.0:1776189706.940849:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7045 00000100:00100000:0.0:1776189706.940852:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f6664e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664393536:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776189706.940868:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189706.940892:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f6664e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664393536:12345-0@lo:13:osp-pre-0-0.0 Request processed in 39us (348us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189706.940898:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7045 00000100:00100000:0.0:1776189706.940935:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064636c08 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664393600:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189706.940949:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a7066b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664393536:0@lo:13:osp-pre-0-0.0 00010000:02020000:2.0:1776189706.996410:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189707.252280:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189707.260373:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189707.508290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189707.764291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189707.964567:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1776189707.964589:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064636c08 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664393728:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189707.964658:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189707.964695:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000761163f9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664393664:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189707.964718:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189707.964752:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000076bff338 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664393792:0@lo:400:kworker/u8:1.0 00000100:00080000:0.0:1776189707.964763:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189707.964768:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189707.964772:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189707.964777:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9849 00000100:00080000:0.0:1776189707.964780:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00100000:3.0:1776189707.964780:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776189707.964786:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189707.964804:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d82e179d x1862459664393920/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189707.964809:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f1c366f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664393856:0@lo:400:kworker/u8:1.0 00000100:00080000:0.0:1776189707.964824:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189707.964827:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189707.964829:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189707.964832:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9849 00000100:00080000:0.0:1776189707.964834:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00100000:3.0:1776189707.964834:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776189707.964837:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189707.964846:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b7fee202 x1862459664394048/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189707.964850:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00080000:0.0:1776189707.964855:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1776189707.964855:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c0bb1b99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664393984:0@lo:400:kworker/u8:1.0 00000100:00080000:0.0:1776189707.964858:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189707.964859:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189707.964862:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9849 00000100:00080000:0.0:1776189707.964864:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 6s 00000100:00080000:0.0:1776189707.964866:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189707.964873:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189707.964876:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000a0e79be x1862459664394112/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189707.964967:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001e60ceac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664394176:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189707.965002:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189707.965022:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776189707.965026:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000270ba913 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664394240:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189707.965047:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189707.965075:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664394176 00000100:00100000:1.0:1776189707.965115:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664394240 00000100:00100000:1.0:1776189707.965150:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8745 00000100:00100000:1.0:1776189707.965163:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007c231a0e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664394176:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189707.965230:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007c231a0e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664394176:12345-0@lo:400:kworker/u8:1.0 Request processed in 74us (227us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189707.965238:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8745 00000100:00100000:1.0:1776189707.965253:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8746 00000100:00100000:1.0:1776189707.965256:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ff5d45b7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664394240:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189707.965282:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ff5d45b7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664394240:12345-0@lo:400:kworker/u8:1.0 Request processed in 25us (235us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189707.965288:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8746 00000100:00100000:1.0:1776189707.965306:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664393728 00000100:00100000:1.0:1776189707.965329:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664393664 00000100:00100000:1.0:1776189707.965347:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7046 00000100:00100000:1.0:1776189707.965350:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000088c2a62e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664393728:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776189707.965363:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189707.965390:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000088c2a62e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664393728:12345-0@lo:13:osp-pre-0-1.0 Request processed in 39us (733us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189707.965395:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7046 00000100:00100000:1.0:1776189707.965405:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7047 00000100:00100000:1.0:1776189707.965408:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d45cc7cd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664393664:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189707.965418:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189707.965441:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d45cc7cd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664393664:12345-0@lo:13:osp-pre-1-1.0 Request processed in 31us (722us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189707.965446:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7047 00000100:00100000:1.0:1776189707.965467:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664393792 00000100:00100000:1.0:1776189707.965492:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1581 00000100:00100000:1.0:1776189707.965496:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000027854250 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664393792:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189707.965525:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000027854250 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664393792:12345-0@lo:400:kworker/u8:1.0 Request processed in 28us (744us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189707.965531:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1581 00000100:00100000:1.0:1776189707.965570:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000270ba913 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664394240:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189707.965580:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001e60ceac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664394176:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189707.965605:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d82e179d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664393920:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189707.965647:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000076bff338 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664393792:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189707.965662:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064636c08 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664393728:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189707.965672:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000761163f9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664393664:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189707.967762:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094837a0a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664394304:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189707.967823:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189707.972024:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664393856 00000100:00100000:3.0:1776189707.972043:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189707.972072:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b7fee202 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664394048:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189707.972081:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664393984 00000100:00100000:0.0:1776189707.972102:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664394304 00000100:00100000:0.0:1776189707.972114:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664393920 00000100:00100000:0.0:1776189707.972128:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10205 00000100:00100000:0.0:1776189707.972146:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd5e571c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664393856:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189707.972219:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd5e571c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664393856:12345-0@lo:400:kworker/u8:1.0 Request processed in 85us (7384us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189707.972226:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10205 00000100:00100000:0.0:1776189707.972243:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10206 00000100:00100000:0.0:1776189707.972248:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005d3cfd23 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664393984:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189707.972268:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10207 00000100:00100000:1.0:1776189707.972275:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cf6b5d65 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664394304:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189707.972280:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005d3cfd23 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664393984:12345-0@lo:400:kworker/u8:1.0 Request processed in 30us (7404us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189707.972288:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10206 00000100:00100000:0.0:1776189707.972302:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57557 00000100:00100000:0.0:1776189707.972306:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003f96cf0d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664393920:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189707.972320:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cf6b5d65 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664394304:12345-0@lo:400:kworker/u8:1.0 Request processed in 45us (4497us total) trans 0 rc 0/0 00010000:02000000:0.0:1776189707.972322:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189707.972353:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003f96cf0d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664393920:12345-0@lo:38:kworker/u8:1.0 Request processed in 46us (314us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189707.972359:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57557 00000100:00100000:1.0:1776189707.972567:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10207 00000100:00100000:2.0:1776189707.972671:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094837a0a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664394304:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189707.972686:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c0bb1b99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664393984:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189707.972695:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f1c366f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664393856:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189707.972773:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189707.972870:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664394048 00000100:00100000:3.0:1776189707.972973:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000a0e79be pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664394112:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189707.973005:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1776189707.973037:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189707.973041:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189707.973046:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d82e179d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664393920:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189707.973049:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664394112 00000100:00100000:0.0:1776189707.973086:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9185 00000100:00100000:0.0:1776189707.973089:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007deebc61 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664394112:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189707.973099:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189707.973146:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007deebc61 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664394112:12345-0@lo:38:kworker/u8:1.0 Request processed in 42us (128us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189707.973153:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9185 00000100:00100000:1.0:1776189707.973293:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57558 00000100:00100000:1.0:1776189707.973300:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000af42fd42 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664394048:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189707.973308:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00080000:3.0:1776189707.973599:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189707.973603:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189707.973606:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000a0e79be pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664394112:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189707.973670:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000af42fd42 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664394048:12345-0@lo:38:kworker/u8:1.0 Request processed in 369us (898us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189707.973678:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57558 00000100:00080000:3.0:1776189707.973744:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189707.973748:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189707.973751:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b7fee202 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664394048:0@lo:38:kworker/u8:1.0 00010000:02020000:2.0:1776189708.020283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189708.277146:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189708.286967:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189708.532309:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189708.788277:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189709.044297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189709.300336:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189709.308517:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189709.556401:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189709.816008:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189710.068292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189710.324381:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189710.337327:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189710.580700:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189710.836498:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189711.092306:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189711.348536:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189711.357336:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189711.604315:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189711.741136:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189711.741215:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189711.742245:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654965760 00000100:00100000:2.0:1776189711.742310:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189711.744398:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1582 00000100:00100000:3.0:1776189711.744407:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000192d1549 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654965760:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1776189711.744497:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000192d1549 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654965760:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 90us (3368us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189711.744505:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1582 00000100:00100000:3.0:1776189711.746908:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654965824 00000100:00100000:3.0:1776189711.746980:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654965888 00000100:00100000:3.0:1776189711.747009:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10208 00000100:00100000:3.0:1776189711.747015:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006bb8567a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654965824:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1776189711.747103:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006bb8567a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654965824:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 87us (5888us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189711.747112:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10208 00000100:00100000:3.0:1776189711.747128:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57559 00000100:00100000:3.0:1776189711.747132:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a02f1c47 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654965888:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:3.0:1776189711.747146:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:3.0:1776189711.747192:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a02f1c47 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2277:x1862459654965888:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 38us (4864us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189711.747199:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57559 00010000:02020000:2.0:1776189711.860392:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189712.060670:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b7fee202 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664394368:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189712.060753:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189712.060852:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664394368 00000100:00100000:3.0:1776189712.060905:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7048 00000100:00100000:3.0:1776189712.060912:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007a388dd2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664394368:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776189712.060943:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189712.061066:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776189712.061076:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c4b16b97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664394432:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776189712.061106:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189712.062422:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664394432 00000100:00100000:0.0:1776189712.062479:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7049 00000100:00100000:0.0:1776189712.062484:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a9575a34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664394432:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776189712.062510:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189712.062811:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a9575a34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664394432:12345-0@lo:13:osp-pre-0-0.0 Request processed in 325us (1704us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189712.062819:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7049 00000100:00100000:0.0:1776189712.062869:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c4b16b97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664394432:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189712.067228:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007a388dd2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664394368:12345-0@lo:13:osp-pre-1-0.0 Request processed in 6307us (6467us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189712.067249:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7048 00000100:00100000:2.0:1776189712.068927:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b7fee202 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664394368:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189712.117609:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189712.372281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189712.381483:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189712.628913:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189712.884284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:2.0:1776189713.085314:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189713.085322:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189713.085326:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189713.085335:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9854 00000100:00080000:2.0:1776189713.085339:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00080000:2.0:1776189713.085344:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189713.085366:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a7368321 x1862459664394624/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189713.085385:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00080000:2.0:1776189713.085393:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189713.085397:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189713.085399:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189713.085402:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9854 00000100:00080000:2.0:1776189713.085405:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00100000:1.0:1776189713.085406:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cf6eb6bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664394496:0@lo:400:kworker/u8:0.0 00000100:00080000:2.0:1776189713.085410:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189713.085423:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b9ada007 x1862459664394752/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189713.085431:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189713.085435:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189713.085436:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189713.085440:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9854 00000100:00080000:2.0:1776189713.085442:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 11s 00000100:00080000:2.0:1776189713.085444:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189713.085456:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a969b6e4 x1862459664394816/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189713.085498:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189713.085530:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b5045f9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664394880:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189713.085532:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000761163f9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664394688:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189713.085566:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189713.085581:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189713.085583:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c6bc9cdf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664395136:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189713.085608:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d52de5d9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664395072:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189713.085614:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189713.085638:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1776189713.085642:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189713.085646:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007f770c5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664394944:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189713.085666:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1776189713.085674:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189713.085692:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d5cfaed8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664395200:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189713.085712:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189713.085740:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664394880 00000100:00100000:1.0:1776189713.085776:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664395136 00000100:00100000:1.0:1776189713.085793:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664395072 00000100:00100000:1.0:1776189713.085806:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664394944 00000100:00100000:1.0:1776189713.085822:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8747 00000100:00100000:1.0:1776189713.085827:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008821baff pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664394880:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189713.085881:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008821baff pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664394880:12345-0@lo:400:kworker/u8:0.0 Request processed in 54us (316us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189713.085890:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8747 00000100:00100000:0.0:1776189713.085904:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8748 00000100:00100000:1.0:1776189713.085912:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8749 00000100:00100000:0.0:1776189713.085913:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e9138b66 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664395136:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189713.085919:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000058c450e1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664395072:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189713.085957:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e9138b66 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664395136:12345-0@lo:400:kworker/u8:0.0 Request processed in 46us (343us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189713.085963:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000058c450e1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664395072:12345-0@lo:400:kworker/u8:0.0 Request processed in 44us (323us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189713.085967:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8748 00000100:00100000:1.0:1776189713.085975:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8749 00000100:00100000:0.0:1776189713.085993:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8750 00000100:00100000:0.0:1776189713.085998:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005355735b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664394944:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189713.086051:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005355735b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664394944:12345-0@lo:400:kworker/u8:0.0 Request processed in 52us (377us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189713.086079:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000053d4c624 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664394560:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189713.086105:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189713.086122:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d4382e95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664395008:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189713.086143:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189713.086198:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d52de5d9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664395072:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189713.086213:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776189713.086220:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000a0e79be pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664395264:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189713.086240:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189713.086268:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664394688 00000100:00100000:2.0:1776189713.086298:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664394560 00000100:00100000:2.0:1776189713.086315:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664395008 00000100:00100000:2.0:1776189713.086331:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10209 00000100:00100000:2.0:1776189713.086335:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005cdf5f34 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664394688:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189713.086370:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005cdf5f34 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664394688:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (793us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189713.086376:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10209 00000100:00100000:2.0:1776189713.086388:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10210 00000100:00100000:2.0:1776189713.086391:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c3185a52 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664394560:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189713.086416:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c3185a52 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664394560:12345-0@lo:400:kworker/u8:0.0 Request processed in 22us (310us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189713.086423:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10210 00000100:00100000:2.0:1776189713.086435:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10211 00000100:00100000:2.0:1776189713.086440:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c22b559c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664395008:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189713.086446:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a7368321 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664394624:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189713.086465:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c22b559c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664395008:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (321us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189713.086472:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10211 00000100:00100000:3.0:1776189713.086493:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189713.086512:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664395200 00000100:00100000:3.0:1776189713.086516:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b9ada007 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664394752:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189713.086539:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664395264 00000100:00100000:3.0:1776189713.086540:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189713.086550:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a969b6e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664394816:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189713.086561:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7050 00000100:00100000:2.0:1776189713.086566:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008dfea28a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664395200:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189713.086577:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:2.0:1776189713.086583:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189713.086612:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664394496 00000100:00100000:2.0:1776189713.086619:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008dfea28a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664395200:12345-0@lo:13:osp-pre-1-1.0 Request processed in 52us (905us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189713.086627:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7050 00000100:00100000:3.0:1776189713.086644:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1583 00000100:00100000:2.0:1776189713.086647:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7051 00000100:00100000:3.0:1776189713.086652:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b64e4d66 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664394496:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189713.086656:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000bb8b061 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664395264:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189713.086679:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189713.086689:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b64e4d66 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664394496:12345-0@lo:400:kworker/u8:0.0 Request processed in 38us (1193us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189713.086697:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1583 00000100:00100000:2.0:1776189713.086713:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000bb8b061 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664395264:12345-0@lo:13:osp-pre-0-1.0 Request processed in 58us (471us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189713.086720:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7051 00000100:00100000:3.0:1776189713.086732:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664394624 00000100:00100000:2.0:1776189713.086747:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664394752 00000100:00100000:3.0:1776189713.086751:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57560 00000100:00100000:3.0:1776189713.086756:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fe79c55d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664394624:12345-0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189713.086766:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57561 00010000:02000000:3.0:1776189713.086770:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189713.086773:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ef7b5ef7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664394752:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189713.086784:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189713.086810:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fe79c55d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664394624:12345-0@lo:38:kworker/u8:0.0 Request processed in 52us (318us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189713.086817:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57560 00000100:00100000:2.0:1776189713.086821:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ef7b5ef7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664394752:12345-0@lo:38:kworker/u8:0.0 Request processed in 47us (280us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189713.086829:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57561 00000100:00100000:3.0:1776189713.086836:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664394816 00000100:00100000:3.0:1776189713.086858:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9186 00000100:00100000:3.0:1776189713.086862:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002e7c1306 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664394816:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189713.086869:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189713.086879:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000a0e79be pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664395264:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189713.086895:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000053d4c624 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664394560:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189713.086897:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002e7c1306 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664394816:12345-0@lo:38:kworker/u8:0.0 Request processed in 32us (319us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189713.086906:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9186 00000100:00100000:2.0:1776189713.086912:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d4382e95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664395008:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189713.086924:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000761163f9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664394688:0@lo:400:kworker/u8:0.0 00000100:00080000:3.0:1776189713.086943:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189713.086948:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189713.086952:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a7368321 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664394624:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776189713.086961:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189713.086964:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189713.086966:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b9ada007 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664394752:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776189713.086974:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776189713.086976:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:3.0:1776189713.086979:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a969b6e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664394816:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189713.087074:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007f770c5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664394944:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189713.087088:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d5cfaed8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664395200:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189713.087097:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b5045f9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664394880:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189713.087104:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c6bc9cdf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664395136:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189713.087112:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cf6eb6bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664394496:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189713.090725:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8750 00010000:02020000:2.0:1776189713.141095:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189713.396927:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189713.406683:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189713.652686:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189713.908289:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189714.164291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189714.420284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189714.428436:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189714.676276:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189714.932928:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189715.188776:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189715.444282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189715.454032:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189715.700285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189715.956283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189716.212304:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189716.468303:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189716.476400:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189716.724360:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189716.858885:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189716.858954:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189716.858996:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189716.859047:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654965952 00000100:00100000:1.0:1776189716.859099:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1584 00000100:00100000:1.0:1776189716.859106:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f37c27d2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654965952:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189716.859206:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f37c27d2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654965952:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 68us (220us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189716.859215:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1584 00000100:00100000:0.0:1776189716.859512:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654966016 00000100:00100000:0.0:1776189716.859562:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654966080 00000100:00100000:0.0:1776189716.859657:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10212 00000100:00100000:0.0:1776189716.859662:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000062c76cbe pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654966016:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189716.859715:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000062c76cbe pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654966016:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 52us (838us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189716.859725:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10212 00000100:00100000:0.0:1776189716.859743:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57562 00000100:00100000:0.0:1776189716.859747:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000e7453c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654966080:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776189716.859765:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189716.859797:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000e7453c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654966080:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 49us (801us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189716.859807:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57562 00010000:02020000:2.0:1776189716.980285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189717.180479:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776189717.180504:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d82e179d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664395328:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189717.180718:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002eebf598 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664395392:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189717.180765:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189717.180893:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664395392 00000100:00100000:3.0:1776189717.180905:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189717.181585:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664395328 00000100:00100000:1.0:1776189717.181673:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7052 00000100:00100000:1.0:1776189717.181679:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000043ae8a46 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664395392:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189717.181707:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189717.181728:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7053 00000100:00100000:2.0:1776189717.181734:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d787fee7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664395328:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189717.181759:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000043ae8a46 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664395392:12345-0@lo:13:osp-pre-0-0.0 Request processed in 78us (993us total) trans 0 rc 0/0 00002000:00000024:2.0:1776189717.181766:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189717.181768:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7052 00000100:00100000:2.0:1776189717.181823:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d787fee7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664395328:12345-0@lo:13:osp-pre-1-0.0 Request processed in 87us (918us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189717.181831:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7053 00000100:00100000:1.0:1776189717.181857:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d82e179d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664395328:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189717.181888:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002eebf598 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664395392:0@lo:13:osp-pre-0-0.0 00010000:02020000:2.0:1776189717.236399:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189717.492288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189717.501201:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189717.748284:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189718.004393:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189718.204382:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776189718.204403:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fd041bfd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664395456:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189718.204438:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f1c366f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664395520:0@lo:13:osp-pre-0-1.0 00000100:00080000:0.0:1776189718.204466:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189718.204472:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189718.204476:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1776189718.204477:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189718.204483:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00080000:0.0:1776189718.204485:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9860 00000100:00080000:0.0:1776189718.204489:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:0.0:1776189718.204495:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189718.204499:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094837a0a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664395648:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189718.204517:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000064636c08 x1862459664395712/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189718.204530:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189718.204534:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664395456 00000100:00100000:1.0:1776189718.204542:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776189718.204552:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1776189718.204555:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00080000:0.0:1776189718.204557:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189718.204560:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1776189718.204565:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c0bb1b99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664395584:0@lo:400:kworker/u8:0.0 00000100:00080000:0.0:1776189718.204566:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9860 00000100:00080000:0.0:1776189718.204571:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00100000:1.0:1776189718.204575:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000076bff338 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664395776:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189718.204582:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664395520 00000100:00080000:0.0:1776189718.204586:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189718.204602:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001e60ceac x1862459664395840/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776189718.204607:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7054 00000100:00100000:2.0:1776189718.204616:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776189718.204620:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1776189718.204621:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e67b9083 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664395456:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189718.204624:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776189718.204629:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189718.204631:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189718.204636:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9860 00000100:00080000:0.0:1776189718.204638:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00080000:0.0:1776189718.204642:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00002000:00000024:3.0:1776189718.204649:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189718.204656:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000270ba913 x1862459664395904/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189718.204670:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664395584 00000100:00100000:1.0:1776189718.204672:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664395648 00000100:00100000:3.0:1776189718.204697:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e67b9083 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664395456:12345-0@lo:13:osp-pre-1-1.0 Request processed in 79us (221us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189718.204707:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7054 00000100:00100000:2.0:1776189718.204714:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1585 00000100:00100000:1.0:1776189718.204718:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664395776 00000100:00100000:2.0:1776189718.204721:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002e45a4e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664395584:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189718.204731:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776189718.204731:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7055 00000100:00100000:3.0:1776189718.204737:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a13b6a88 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664395520:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189718.204743:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b7fee202 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664396096:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189718.204748:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10213 00002000:00000024:3.0:1776189718.204750:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189718.204755:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003e8768bb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664395648:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189718.204766:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002e45a4e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664395584:12345-0@lo:400:kworker/u8:0.0 Request processed in 44us (156us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189718.204775:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1585 00000100:00100000:0.0:1776189718.204789:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189718.204810:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664396096 00000100:00100000:2.0:1776189718.204834:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10214 00000100:00100000:2.0:1776189718.204838:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ac2fe2ac pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664395776:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189718.207219:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003e8768bb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664395648:12345-0@lo:400:kworker/u8:0.0 Request processed in 2456us (2671us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189718.207235:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10213 00000100:00100000:1.0:1776189718.207272:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10215 00000100:00100000:1.0:1776189718.207278:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008a7066b0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664396096:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189718.207340:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008a7066b0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664396096:12345-0@lo:400:kworker/u8:0.0 Request processed in 63us (2553us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189718.207346:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10215 00000100:00100000:1.0:1776189718.207419:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d82e179d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664395968:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189718.207453:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189718.207491:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c0bb1b99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664395584:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189718.207514:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f1c366f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664395520:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189718.207539:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776189718.207545:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000a0e79be pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664396032:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189718.207567:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189718.207618:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fd041bfd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664395456:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189718.207652:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664395968 00000100:00100000:1.0:1776189718.207682:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664396032 00000100:00100000:1.0:1776189718.207698:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8751 00000100:00100000:1.0:1776189718.207703:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000058fbcadb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664395968:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189718.207735:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000058fbcadb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664395968:12345-0@lo:400:kworker/u8:0.0 Request processed in 32us (281us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189718.207741:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8751 00000100:00100000:1.0:1776189718.207752:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8752 00000100:00100000:1.0:1776189718.207755:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002eebf598 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664396032:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189718.207780:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002eebf598 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664396032:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (212us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189718.207785:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8752 00000100:00100000:1.0:1776189718.207810:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000a0e79be pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664396032:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189718.207817:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d82e179d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664395968:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189718.207858:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064636c08 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664395712:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189718.207883:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189718.207911:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001e60ceac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664395840:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189718.207939:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189718.207952:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000270ba913 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664395904:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189718.207974:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189718.208014:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664395712 00000100:00100000:1.0:1776189718.208030:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664395840 00000100:00100000:1.0:1776189718.208039:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57563 00000100:00100000:1.0:1776189718.208042:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c4b16b97 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664395712:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189718.208054:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189718.208672:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c4b16b97 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664395712:12345-0@lo:38:kworker/u8:0.0 Request processed in 626us (788us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189718.208680:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57563 00000100:00100000:1.0:1776189718.208696:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57564 00000100:00100000:1.0:1776189718.208699:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d82e179d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664395840:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189718.208707:0:9639:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189718.208729:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d82e179d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3360:x1862459664395840:12345-0@lo:38:kworker/u8:0.0 Request processed in 28us (789us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189718.208734:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57564 00000100:00080000:1.0:1776189718.208762:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189718.208766:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189718.208770:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064636c08 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664395712:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776189718.208778:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189718.208780:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189718.208782:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001e60ceac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664395840:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189718.208847:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664395904 00000100:00100000:1.0:1776189718.208874:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9187 00000100:00100000:1.0:1776189718.208877:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000a0e79be pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664395904:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189718.208884:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189718.209196:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000a0e79be pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664395904:12345-0@lo:38:kworker/u8:0.0 Request processed in 316us (1221us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189718.209202:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9187 00000100:00080000:1.0:1776189718.209228:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189718.209231:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189718.209234:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000270ba913 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664395904:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189718.209775:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b7fee202 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664396096:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189718.209792:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094837a0a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664395648:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189718.212087:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ac2fe2ac pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664395776:12345-0@lo:400:kworker/u8:0.0 Request processed in 7240us (7461us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189718.212105:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10214 00000100:00100000:0.0:1776189718.212246:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000076bff338 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664395776:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189718.212373:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a13b6a88 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664395520:12345-0@lo:13:osp-pre-0-1.0 Request processed in 7634us (7842us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189718.212384:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7055 00010000:02020000:2.0:1776189718.260290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189718.520298:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189718.528143:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189718.772349:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189719.028720:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189719.300295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189719.548742:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189719.556332:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189719.812389:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189720.068297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189720.324286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189720.572476:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189720.580416:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189720.836570:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189721.092286:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189721.348378:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189721.596638:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189721.606209:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189721.860285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189721.978929:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189721.978993:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189721.979436:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189721.979666:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654966208 00000100:00100000:0.0:1776189721.979715:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654966272 00000100:00100000:0.0:1776189721.979732:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10216 00000100:00100000:0.0:1776189721.979738:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009b5045f9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654966208:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189721.979812:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009b5045f9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654966208:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 73us (892us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189721.979821:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57565 00000100:00100000:3.0:1776189721.979827:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a969b6e4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654966272:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00000100:00100000:0.0:1776189721.979840:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10216 00010000:02000000:3.0:1776189721.979855:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189721.981881:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654966144 00000100:00100000:1.0:1776189721.981950:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1586 00000100:00100000:1.0:1776189721.981957:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002ea0417b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654966144:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189721.982036:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002ea0417b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2281:x1862459654966144:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 79us (3043us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189721.982044:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1586 00000100:00100000:3.0:1776189722.004791:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a969b6e4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654966272:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 24951us (25344us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189722.004806:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57565 00010000:02020000:2.0:1776189722.116718:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189722.302258:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000270ba913 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664396160:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189722.302328:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189722.302685:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776189722.302695:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f61bb34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664396224:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776189722.302723:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189722.302933:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664396160 00000100:00100000:0.0:1776189722.302983:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664396224 00000100:00100000:0.0:1776189722.303010:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7056 00000100:00100000:0.0:1776189722.303016:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006d50a26f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664396160:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776189722.303045:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189722.303088:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006d50a26f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664396160:12345-0@lo:13:osp-pre-0-0.0 Request processed in 72us (760us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189722.303096:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7056 00000100:00100000:0.0:1776189722.303112:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7057 00000100:00100000:0.0:1776189722.303116:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b9ada007 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664396224:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189722.303129:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189722.303154:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b9ada007 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664396224:12345-0@lo:13:osp-pre-1-0.0 Request processed in 38us (431us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189722.303159:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7057 00000100:00100000:0.0:1776189722.303225:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000270ba913 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664396160:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776189722.303333:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f61bb34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664396224:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189722.372282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189722.623912:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189722.628290:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189722.884401:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189723.140336:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189723.324465:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00080000:1.0:1776189723.324468:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189723.324475:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189723.324478:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189723.324486:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9865 00000100:00100000:0.0:1776189723.324487:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064636c08 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664396416:0@lo:400:kworker/u8:0.0 00000100:00080000:1.0:1776189723.324492:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189723.324498:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189723.324519:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f1c366f3 x1862459664396480/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189723.324546:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189723.324550:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1776189723.324555:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776189723.324564:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1776189723.324564:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c0bb1b99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664396544:0@lo:400:kworker/u8:0.0 00000100:00080000:1.0:1776189723.324572:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9865 00000100:00080000:1.0:1776189723.324574:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189723.324579:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189723.324588:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001e60ceac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664396352:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189723.324594:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000270ba913 x1862459664396672/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189723.324604:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1776189723.324605:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776189723.324609:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189723.324611:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189723.324615:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9865 00000100:00080000:1.0:1776189723.324618:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1776189723.324620:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776189723.324625:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189723.324633:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1776189723.324642:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c6f4c7a3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664396288:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189723.324651:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000076bff338 x1862459664396736/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189723.324667:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664396416 00000100:00100000:2.0:1776189723.324677:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189723.324702:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018a48c26 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664396608:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189723.324717:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664396544 00000100:00100000:2.0:1776189723.324728:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189723.324739:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10217 00000100:00100000:0.0:1776189723.324744:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e5ef59b0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664396416:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189723.324750:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000412a95e8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664396928:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189723.324769:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10218 00000100:00100000:1.0:1776189723.324776:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000051f898a5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664396544:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189723.324782:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189723.324794:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e5ef59b0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664396416:12345-0@lo:400:kworker/u8:0.0 Request processed in 48us (241us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189723.324811:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10217 00000100:00100000:2.0:1776189723.324818:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776189723.324826:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e42142fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664396992:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189723.324832:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664396928 00000100:00100000:1.0:1776189723.324841:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000051f898a5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664396544:12345-0@lo:400:kworker/u8:0.0 Request processed in 64us (238us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189723.324848:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10218 00000100:00100000:0.0:1776189723.324856:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10219 00000100:00100000:0.0:1776189723.324861:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fd041bfd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664396928:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189723.324867:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189723.324901:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fd041bfd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664396928:12345-0@lo:400:kworker/u8:0.0 Request processed in 38us (123us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189723.324915:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10219 00000100:00100000:2.0:1776189723.324921:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c0bb1b99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664396544:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189723.324937:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f1c366f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664396480:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189723.324949:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000412a95e8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664396928:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189723.324968:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189723.324977:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664396992 00000100:00100000:0.0:1776189723.324989:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000270ba913 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664396672:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189723.325011:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8753 00000100:00100000:0.0:1776189723.325015:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189723.325019:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cf6eb6bb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664396992:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189723.325057:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cf6eb6bb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664396992:12345-0@lo:400:kworker/u8:0.0 Request processed in 39us (192us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189723.325062:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8753 00000100:00100000:2.0:1776189723.325084:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664396352 00000100:00100000:2.0:1776189723.325106:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1587 00000100:00100000:2.0:1776189723.325110:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a9501f9f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664396352:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189723.325136:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a9501f9f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3362:x1862459664396352:12345-0@lo:400:kworker/u8:0.0 Request processed in 25us (518us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189723.325144:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1587 00000100:00100000:2.0:1776189723.325204:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e42142fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664396992:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189723.325220:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b7fee202 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664396864:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189723.325242:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189723.325261:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064636c08 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664396416:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189723.325269:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001e60ceac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3362:1862459664396352:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189723.325281:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776189723.325285:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094837a0a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664396800:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189723.325303:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189723.325322:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664396864 00000100:00100000:2.0:1776189723.325340:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664396800 00000100:00100000:2.0:1776189723.325371:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8754 00000100:00100000:2.0:1776189723.325374:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a7368321 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664396864:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189723.325412:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a7368321 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664396864:12345-0@lo:400:kworker/u8:0.0 Request processed in 36us (170us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189723.325417:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8754 00000100:00100000:2.0:1776189723.325435:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8755 00000100:00100000:2.0:1776189723.325438:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000761163f9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664396800:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189723.325467:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000761163f9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664396800:12345-0@lo:400:kworker/u8:0.0 Request processed in 28us (164us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189723.325472:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8755 00000100:00100000:2.0:1776189723.325502:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094837a0a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664396800:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189723.325510:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b7fee202 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664396864:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189723.325536:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664396288 00000100:00100000:2.0:1776189723.325562:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664396608 00000100:00100000:2.0:1776189723.325578:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7058 00000100:00100000:2.0:1776189723.325581:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000eaae2e6e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664396288:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189723.325594:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189723.325624:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000eaae2e6e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664396288:12345-0@lo:13:osp-pre-0-1.0 Request processed in 41us (948us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189723.325630:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7058 00000100:00100000:2.0:1776189723.325648:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7059 00000100:00100000:2.0:1776189723.325651:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001abd0c8c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664396608:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189723.325660:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189723.325688:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001abd0c8c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664396608:12345-0@lo:13:osp-pre-1-1.0 Request processed in 35us (962us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189723.325693:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7059 00000100:00100000:2.0:1776189723.325730:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c6f4c7a3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664396288:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189723.325749:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018a48c26 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664396608:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189723.326541:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000076bff338 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664396736:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189723.326576:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189723.326929:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664396480 00000100:00100000:0.0:1776189723.326930:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664396672 00000100:00100000:0.0:1776189723.326972:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57566 00000100:00100000:0.0:1776189723.326977:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009257ba59 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664396480:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189723.326991:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189723.327032:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009257ba59 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664396480:12345-0@lo:38:kworker/u8:0.0 Request processed in 54us (2065us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189723.327233:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57567 00000100:00100000:0.0:1776189723.327241:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57566 00000100:00100000:3.0:1776189723.327244:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e071bb1b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664396672:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189723.327254:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00080000:0.0:1776189723.327281:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189723.327285:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189723.327290:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f1c366f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664396480:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189723.327326:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664396736 00000100:00100000:0.0:1776189723.327348:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9188 00000100:00100000:0.0:1776189723.327351:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009fddac3a pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664396736:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189723.327358:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189723.329053:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e071bb1b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664396672:12345-0@lo:38:kworker/u8:0.0 Request processed in 1810us (4037us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189723.329060:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009fddac3a pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664396736:12345-0@lo:38:kworker/u8:0.0 Request processed in 1704us (2481us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189723.329075:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9188 00000100:00100000:3.0:1776189723.329075:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57567 00000100:00080000:0.0:1776189723.329142:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189723.329146:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189723.329149:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000270ba913 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664396672:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776189723.329158:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189723.329161:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189723.329163:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000076bff338 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664396736:0@lo:38:kworker/u8:0.0 00010000:02020000:2.0:1776189723.396296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189723.644378:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189723.652292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189723.908282:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189724.164257:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189724.420281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189724.668595:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189724.676325:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189724.932714:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189725.188320:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189725.444269:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189725.692383:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189725.700318:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189725.957203:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189726.212271:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189726.468849:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189726.716427:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189726.724918:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189726.980919:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189727.102405:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189727.102463:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189727.102496:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189727.102606:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654966464 00000100:00100000:1.0:1776189727.102607:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654966336 00000100:00100000:1.0:1776189727.102653:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1588 00000100:00100000:1.0:1776189727.102661:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004c8f754f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654966336:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1776189727.102664:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654966400 00000100:00100000:1.0:1776189727.102742:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004c8f754f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654966336:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 81us (278us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189727.102747:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57568 00000100:00100000:1.0:1776189727.102759:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1588 00000100:00100000:0.0:1776189727.102760:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c6f4c7a3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654966464:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:0.0:1776189727.102777:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189727.102812:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c6f4c7a3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654966464:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 56us (415us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189727.102819:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57568 00000100:00100000:3.0:1776189727.102934:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10220 00000100:00100000:3.0:1776189727.102938:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000f61bb34 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654966400:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1776189727.102985:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000f61bb34 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654966400:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 46us (489us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189727.102993:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10220 00010000:02020000:2.0:1776189727.236292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189727.422380:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f8f9cfc6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664397056:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189727.422462:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189727.422504:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776189727.422510:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02fb24b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664397120:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189727.422531:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189727.422711:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664397056 00000100:00100000:3.0:1776189727.422767:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664397120 00000100:00100000:3.0:1776189727.422786:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7060 00000100:00100000:3.0:1776189727.422792:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053d4c624 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664397056:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776189727.422821:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189727.422858:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053d4c624 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664397056:12345-0@lo:13:osp-pre-1-0.0 Request processed in 65us (397us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189727.422865:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7060 00000100:00100000:3.0:1776189727.422883:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7061 00000100:00100000:3.0:1776189727.422887:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000018a48c26 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664397120:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189727.422902:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189727.422925:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000018a48c26 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664397120:12345-0@lo:13:osp-pre-0-0.0 Request processed in 37us (394us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189727.422931:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7061 00000100:00100000:2.0:1776189727.423130:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02fb24b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664397120:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189727.423145:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f8f9cfc6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664397056:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189727.492285:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189727.744597:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189727.748269:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189728.004815:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189728.260280:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189728.445424:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776189728.445447:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bf60fbf3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664397184:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189728.445519:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189728.448549:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664397184 00000100:00100000:2.0:1776189728.448624:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7062 00000100:00100000:2.0:1776189728.448630:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b7a82fb0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664397184:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189728.448659:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189728.448752:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b7a82fb0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664397184:12345-0@lo:13:osp-pre-0-1.0 Request processed in 121us (3233us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189728.448759:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7062 00000100:00100000:1.0:1776189728.449031:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bf60fbf3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664397184:0@lo:13:osp-pre-0-1.0 00000100:00080000:2.0:1776189728.449318:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189728.449323:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189728.449325:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189728.449330:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9870 00000100:00080000:2.0:1776189728.449332:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776189728.449336:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189728.449354:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f1c366f3 x1862459664397376/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189728.449383:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189728.449387:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189728.449388:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189728.449391:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9870 00000100:00080000:2.0:1776189728.449393:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776189728.449396:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189728.449406:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000094837a0a x1862459664397504/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189728.449413:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189728.449417:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189728.449418:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189728.449421:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9870 00000100:00080000:2.0:1776189728.449423:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776189728.449425:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189728.449434:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001e60ceac x1862459664397568/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189728.449480:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b7fee202 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664397440:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189728.449680:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064636c08 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664397632:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189728.449730:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189728.449755:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1776189728.449760:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000076bff338 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664397248:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189728.449785:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189728.449810:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c0bb1b99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664397696:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189728.449832:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189728.449859:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664397632 00000100:00100000:1.0:1776189728.449888:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664397696 00000100:00100000:1.0:1776189728.449915:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8756 00000100:00100000:1.0:1776189728.449919:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007a543fbe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664397632:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189728.449957:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007a543fbe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664397632:12345-0@lo:400:kworker/u8:0.0 Request processed in 36us (228us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189728.449963:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8756 00000100:00100000:1.0:1776189728.449976:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8757 00000100:00100000:1.0:1776189728.449979:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000008468a4e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664397696:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189728.450004:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000008468a4e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664397696:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (172us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189728.450010:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8757 00000100:00100000:1.0:1776189728.450028:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664397248 00000100:00100000:1.0:1776189728.450049:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1589 00000100:00100000:1.0:1776189728.450053:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a9f52f65 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664397248:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189728.450078:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a9f52f65 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664397248:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (293us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189728.450083:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1589 00000100:00100000:1.0:1776189728.450129:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000076bff338 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664397248:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189728.450605:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c0bb1b99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664397696:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189728.450615:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064636c08 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664397632:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189728.450662:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f1c366f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664397376:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189728.450703:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189728.450722:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094837a0a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664397504:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189728.450743:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189728.450757:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001e60ceac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664397568:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189728.450779:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189728.451261:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189728.451280:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d5cfaed8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664397824:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189728.451302:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189728.451321:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1776189728.451327:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000270ba913 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664397312:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189728.451347:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189728.451360:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c6bc9cdf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664397760:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189728.451382:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189728.451413:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664397376 00000100:00100000:2.0:1776189728.451437:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664397504 00000100:00100000:2.0:1776189728.451445:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664397440 00000100:00100000:2.0:1776189728.451459:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664397312 00000100:00100000:2.0:1776189728.451468:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664397760 00000100:00100000:2.0:1776189728.451478:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10221 00000100:00100000:2.0:1776189728.451483:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007f770c5d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664397440:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189728.451644:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10222 00000100:00100000:3.0:1776189728.451648:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d52de5d9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664397312:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189728.451700:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d52de5d9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664397312:12345-0@lo:400:kworker/u8:0.0 Request processed in 41us (344us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189728.451707:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10222 00000100:00100000:3.0:1776189728.451723:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10223 00000100:00100000:3.0:1776189728.451726:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000033264082 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664397760:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189728.451751:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000033264082 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664397760:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (369us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189728.451757:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10223 00000100:00100000:3.0:1776189728.451768:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57569 00000100:00100000:3.0:1776189728.451771:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000748b663e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664397376:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189728.451784:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189728.451824:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000748b663e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664397376:12345-0@lo:38:kworker/u8:0.0 Request processed in 51us (1131us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189728.451830:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57569 00000100:00100000:3.0:1776189728.451842:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57570 00000100:00100000:3.0:1776189728.451846:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e42142fc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664397504:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189728.451852:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189728.451874:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e42142fc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664397504:12345-0@lo:38:kworker/u8:0.0 Request processed in 27us (1130us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189728.451879:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57570 00000100:00100000:3.0:1776189728.452288:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000270ba913 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664397312:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189728.452300:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c6bc9cdf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664397760:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189728.452357:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664397568 00000100:00100000:3.0:1776189728.452382:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9189 00000100:00100000:3.0:1776189728.452385:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000412a95e8 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664397568:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776189728.452393:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189728.452420:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000412a95e8 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664397568:12345-0@lo:38:kworker/u8:0.0 Request processed in 34us (1641us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189728.452426:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9189 00000100:00100000:3.0:1776189728.452451:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664397824 00000100:00100000:3.0:1776189728.452472:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8758 00000100:00100000:3.0:1776189728.452476:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d4382e95 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664397824:12345-0@lo:400:kworker/u8:0.0 00000100:00080000:1.0:1776189728.452604:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189728.452608:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189728.452613:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f1c366f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664397376:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776189728.452621:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189728.452623:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189728.452625:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094837a0a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664397504:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776189728.452632:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189728.452634:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189728.452636:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001e60ceac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664397568:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189728.453184:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007f770c5d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664397440:12345-0@lo:400:kworker/u8:0.0 Request processed in 1685us (1909us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189728.453194:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10221 00000100:00100000:3.0:1776189728.453342:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d4382e95 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664397824:12345-0@lo:400:kworker/u8:0.0 Request processed in 864us (2039us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189728.453349:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8758 00000100:00100000:1.0:1776189728.454543:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b7fee202 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664397440:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189728.454559:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d5cfaed8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664397824:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189728.455080:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776189728.455093:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c094ed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664397888:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189728.455256:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189728.455382:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664397888 00000100:00100000:0.0:1776189728.455427:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7063 00000100:00100000:0.0:1776189728.455432:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053ba7e65 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664397888:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189728.455455:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189728.455509:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053ba7e65 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664397888:12345-0@lo:13:osp-pre-1-1.0 Request processed in 76us (254us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189728.455516:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7063 00000100:00100000:3.0:1776189728.455681:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c094ed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664397888:0@lo:13:osp-pre-1-1.0 00010000:02020000:2.0:1776189728.516403:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189728.767111:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189728.772430:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189729.029269:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189729.284467:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189729.540325:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189729.788948:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189729.797316:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189730.052280:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189730.308277:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189730.567877:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189730.813153:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189730.822216:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189731.076328:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189731.332280:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189731.588333:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189731.838453:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189731.844328:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189732.100281:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189732.218341:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189732.218457:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189732.218640:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654966528 00000100:00100000:1.0:1776189732.218724:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1590 00000100:00100000:1.0:1776189732.218731:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000be36d03e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654966528:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776189732.218800:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000be36d03e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654966528:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 68us (342us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189732.218807:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1590 00000100:00100000:2.0:1776189732.218904:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654966592 00000100:00100000:2.0:1776189732.218953:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10224 00000100:00100000:2.0:1776189732.218958:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000041c094ed pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654966592:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189732.218966:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189732.219113:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000041c094ed pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654966592:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 154us (790us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189732.219122:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10224 00000100:00100000:2.0:1776189732.219136:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654966656 00000100:00100000:2.0:1776189732.219156:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57571 00000100:00100000:2.0:1776189732.219159:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022f2aa27 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654966656:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776189732.219205:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189732.244559:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022f2aa27 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2277:x1862459654966656:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 25390us (25590us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189732.244583:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57571 00010000:02020000:2.0:1776189732.356803:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189732.540475:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776189732.540500:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f8f9cfc6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664397952:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189732.540596:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189732.540751:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664397952 00000100:00100000:0.0:1776189732.540802:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7064 00000100:00100000:0.0:1776189732.540808:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d72c99d5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664397952:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189732.540834:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189732.540871:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d72c99d5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664397952:12345-0@lo:13:osp-pre-1-0.0 Request processed in 63us (276us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189732.540878:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7064 00000100:00100000:2.0:1776189732.540999:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f8f9cfc6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664397952:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189732.541198:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000067c9d69b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664398016:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189732.541222:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189732.541282:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664398016 00000100:00100000:0.0:1776189732.541316:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7065 00000100:00100000:0.0:1776189732.541320:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba6916cf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664398016:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776189732.541334:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189732.541382:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba6916cf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664398016:12345-0@lo:13:osp-pre-0-0.0 Request processed in 61us (160us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189732.541387:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7065 00000100:00100000:3.0:1776189732.541517:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000067c9d69b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664398016:0@lo:13:osp-pre-0-0.0 00010000:02020000:2.0:1776189732.612371:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189732.860686:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189732.868322:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189733.126051:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189733.380796:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189733.564705:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b7fee202 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664398080:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189733.564775:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1776189733.564822:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189733.564828:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189733.564831:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189733.564837:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9875 00000100:00100000:2.0:1776189733.564840:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664398080 00000100:00080000:3.0:1776189733.564840:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776189733.564847:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189733.564866:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bf60fbf3 x1862459664398272/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189733.564890:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7066 00000100:00080000:3.0:1776189733.564892:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189733.564896:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:2.0:1776189733.564897:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000013c0eaff pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664398080:12345-0@lo:13:osp-pre-0-1.0 00000100:00080000:3.0:1776189733.564899:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189733.564905:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9875 00000100:00080000:3.0:1776189733.564907:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776189733.564910:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00002000:00000024:2.0:1776189733.564921:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189733.564923:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d5cfaed8 x1862459664398400/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776189733.564933:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776189733.564936:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1776189733.564937:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776189733.564941:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9875 00000100:00080000:3.0:1776189733.564942:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776189733.564946:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:3.0:1776189733.564959:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c6bc9cdf x1862459664398464/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189733.564966:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000013c0eaff pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664398080:12345-0@lo:13:osp-pre-0-1.0 Request processed in 67us (190us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189733.564982:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7066 00000100:00100000:2.0:1776189733.565012:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd150e5b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664398528:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189733.565040:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009820cd1e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664398208:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189733.565062:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189733.565078:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189733.565100:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009015c790 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664398656:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189733.565109:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b7fee202 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664398080:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189733.565123:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189733.565132:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1776189733.565140:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02fb24b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664398336:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189733.565150:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1776189733.565158:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000067c9d69b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664398144:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189733.565190:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189733.565212:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b385e94b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664398720:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189733.565219:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189733.565241:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005bd91e44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664398592:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189733.565246:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189733.565291:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664398528 00000100:00100000:2.0:1776189733.565314:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664398720 00000100:00100000:2.0:1776189733.565329:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8759 00000100:00100000:2.0:1776189733.565333:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007ec15c7b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664398528:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189733.565368:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007ec15c7b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664398528:12345-0@lo:400:kworker/u8:1.0 Request processed in 33us (308us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189733.565374:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8759 00000100:00100000:2.0:1776189733.565385:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8760 00000100:00100000:2.0:1776189733.565389:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b8bc9a84 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664398720:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189733.565431:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b8bc9a84 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664398720:12345-0@lo:400:kworker/u8:1.0 Request processed in 41us (189us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189733.565522:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664398208 00000100:00100000:0.0:1776189733.565561:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664398656 00000100:00100000:0.0:1776189733.565578:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664398336 00000100:00100000:0.0:1776189733.565604:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10225 00000100:00100000:0.0:1776189733.565607:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e310c888 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664398208:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189733.565643:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e310c888 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3364:x1862459664398208:12345-0@lo:400:kworker/u8:1.0 Request processed in 35us (568us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189733.565649:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10225 00000100:00100000:0.0:1776189733.565660:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10226 00000100:00100000:0.0:1776189733.565663:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009b0629a6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664398656:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189733.565687:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009b0629a6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3364:x1862459664398656:12345-0@lo:400:kworker/u8:1.0 Request processed in 23us (566us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189733.565694:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10226 00000100:00100000:0.0:1776189733.565714:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10227 00000100:00100000:0.0:1776189733.565718:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b25bee14 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664398336:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189733.565742:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b25bee14 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664398336:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (572us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189733.565747:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10227 00000100:00100000:2.0:1776189733.565756:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8760 00000100:00100000:0.0:1776189733.565794:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664398144 00000100:00100000:2.0:1776189733.565795:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02fb24b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664398336:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189733.565808:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b385e94b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664398720:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189733.565821:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd150e5b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664398528:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189733.565825:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1591 00000100:00100000:0.0:1776189733.565830:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000816fca97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664398144:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189733.565867:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000816fca97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3364:x1862459664398144:12345-0@lo:400:kworker/u8:1.0 Request processed in 35us (648us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189733.565874:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1591 00000100:00100000:1.0:1776189733.567598:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bf60fbf3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664398272:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189733.567657:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189733.567684:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d5cfaed8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664398400:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189733.567704:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189733.567716:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c6bc9cdf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664398464:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776189733.567736:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189733.567793:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664398464 00000100:00100000:1.0:1776189733.567825:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9190 00000100:00100000:1.0:1776189733.567830:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000270ba913 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664398464:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776189733.567843:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189733.567880:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000270ba913 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664398464:12345-0@lo:38:kworker/u8:1.0 Request processed in 49us (143us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189733.567886:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9190 00000100:00080000:1.0:1776189733.567921:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189733.567925:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189733.567929:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c6bc9cdf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664398464:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189733.568140:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664398272 00000100:00100000:0.0:1776189733.568169:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664398400 00000100:00100000:0.0:1776189733.568206:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57572 00000100:00100000:0.0:1776189733.568209:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000094837a0a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664398272:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189733.568217:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189733.568246:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000094837a0a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664398272:12345-0@lo:38:kworker/u8:1.0 Request processed in 36us (589us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189733.568252:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57572 00000100:00100000:2.0:1776189733.568328:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001e60ceac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664398784:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189733.568361:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189733.568395:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664398784 00000100:00100000:2.0:1776189733.568445:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7067 00000100:00100000:2.0:1776189733.568449:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053ddeb10 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664398784:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189733.568479:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189733.568511:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053ddeb10 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664398784:12345-0@lo:13:osp-pre-1-1.0 Request processed in 61us (150us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189733.568517:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7067 00000100:00100000:2.0:1776189733.569079:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001e60ceac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664398784:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189733.569081:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189733.569116:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57573 00000100:00100000:3.0:1776189733.569117:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009820cd1e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3364:1862459664398208:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189733.569123:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f1c366f3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664398400:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776189733.569134:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009015c790 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3364:1862459664398656:0@lo:400:kworker/u8:1.0 00010000:02000000:2.0:1776189733.569136:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189733.569162:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000067c9d69b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3364:1862459664398144:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189733.569204:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f1c366f3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664398400:12345-0@lo:38:kworker/u8:1.0 Request processed in 81us (1497us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189733.569212:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57573 00000100:00100000:1.0:1776189733.569335:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664398592 00000100:00100000:1.0:1776189733.569374:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8761 00000100:00100000:1.0:1776189733.569377:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b385e94b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664398592:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776189733.569428:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b385e94b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664398592:12345-0@lo:400:kworker/u8:1.0 Request processed in 49us (350us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189733.569434:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8761 00000100:00080000:1.0:1776189733.569465:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189733.569469:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189733.569471:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bf60fbf3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664398272:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776189733.569480:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189733.569483:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189733.569485:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d5cfaed8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664398400:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189733.570718:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005bd91e44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664398592:0@lo:400:kworker/u8:1.0 00010000:02020000:2.0:1776189733.639003:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189733.885478:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189733.892832:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189734.150740:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189734.404275:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189734.660459:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189734.908385:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189734.916296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189735.174037:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189735.428420:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189735.684782:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189735.933776:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189735.940295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189736.196691:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189736.454972:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189736.708314:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189736.956469:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189736.964331:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189737.220346:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189737.339317:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189737.339367:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189737.339405:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189737.339520:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654966784 00000100:00100000:2.0:1776189737.339576:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654966848 00000100:00100000:2.0:1776189737.339598:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10228 00000100:00100000:2.0:1776189737.339607:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009015c790 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654966784:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776189737.339677:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009015c790 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654966784:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 72us (367us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189737.339686:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10228 00000100:00100000:2.0:1776189737.339703:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57574 00000100:00100000:2.0:1776189737.339707:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005bd91e44 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654966848:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:2.0:1776189737.339721:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189737.339752:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005bd91e44 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654966848:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 45us (348us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189737.339759:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57574 00000100:00100000:1.0:1776189737.340097:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654966720 00000100:00100000:1.0:1776189737.340135:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1592 00000100:00100000:1.0:1776189737.340140:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c4f71e57 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654966720:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776189737.340211:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c4f71e57 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654966720:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 70us (844us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189737.340218:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1592 00010000:02020000:2.0:1776189737.476413:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189737.660521:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a829435f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664398848:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189737.660588:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189737.660683:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664398848 00000100:00100000:3.0:1776189737.660732:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7068 00000100:00100000:3.0:1776189737.660738:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000087513f3f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664398848:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189737.660767:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189737.660818:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000087513f3f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664398848:12345-0@lo:13:osp-pre-0-0.0 Request processed in 80us (231us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189737.660825:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7068 00000100:00100000:2.0:1776189737.661190:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a829435f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664398848:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189737.662385:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776189737.662401:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064636c08 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664398912:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189737.662444:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189737.663406:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664398912 00000100:00100000:1.0:1776189737.663460:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7069 00000100:00100000:1.0:1776189737.663466:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e6402bd2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664398912:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189737.663502:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189737.663571:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e6402bd2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664398912:12345-0@lo:13:osp-pre-1-0.0 Request processed in 105us (1127us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189737.663577:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7069 00000100:00100000:3.0:1776189737.663634:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064636c08 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664398912:0@lo:13:osp-pre-1-0.0 00010000:02020000:2.0:1776189737.732358:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189737.992917:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189737.993026:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189738.244434:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189738.500603:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:1.0:1776189738.684598:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189738.684605:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189738.684608:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189738.684616:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9880 00000100:00080000:1.0:1776189738.684621:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00080000:1.0:1776189738.684625:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189738.684647:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000064636c08 x1862459664399104/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189738.684678:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189738.684681:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189738.684682:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189738.684686:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9880 00000100:00080000:1.0:1776189738.684688:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00080000:1.0:1776189738.684691:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189738.684700:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b7fee202 x1862459664399232/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189738.684709:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189738.684711:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189738.684713:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189738.684716:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9880 00000100:00080000:1.0:1776189738.684717:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 6s 00000100:00080000:1.0:1776189738.684720:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189738.684731:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d5cfaed8 x1862459664399296/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189738.684766:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c6bc9cdf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664399360:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189738.684813:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064636c08 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664399104:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189738.684871:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189738.684898:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b7fee202 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664399232:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189738.684923:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189738.684943:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d5cfaed8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664399296:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189738.684972:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189738.685013:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664399296 00000100:00100000:1.0:1776189738.685026:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000076bff338 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664399040:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189738.685045:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9191 00000100:00100000:0.0:1776189738.685049:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189738.685055:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009ed5190d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664399296:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189738.685071:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189738.685115:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c0bb1b99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664398976:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189738.685389:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000050a6a110 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664399616:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189738.685416:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189738.685434:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1776189738.685439:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001e60ceac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664399168:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189738.685462:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189738.685497:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d6bc8daa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664399552:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189738.685519:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189738.685550:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664399104 00000100:00100000:0.0:1776189738.685576:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664399232 00000100:00100000:2.0:1776189738.685584:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009ed5190d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664399296:12345-0@lo:38:kworker/u8:0.0 Request processed in 528us (612us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189738.685593:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664399168 00000100:00100000:0.0:1776189738.685617:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10229 00000100:00100000:0.0:1776189738.685621:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004ef5af0c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664399168:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189738.685663:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004ef5af0c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3364:x1862459664399168:12345-0@lo:400:kworker/u8:0.0 Request processed in 41us (201us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189738.685671:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10229 00000100:00100000:0.0:1776189738.685685:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57575 00000100:00100000:0.0:1776189738.685688:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002ab97433 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664399104:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189738.685697:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189738.685721:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002ab97433 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664399104:12345-0@lo:38:kworker/u8:0.0 Request processed in 32us (851us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189738.685728:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57575 00000100:00100000:0.0:1776189738.685739:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57576 00000100:00100000:0.0:1776189738.685741:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000e05519f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664399232:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189738.685762:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189738.685785:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000e05519f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664399232:12345-0@lo:38:kworker/u8:0.0 Request processed in 42us (863us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189738.685791:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57576 00000100:00100000:0.0:1776189738.685829:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001e60ceac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3364:1862459664399168:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189738.685867:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664399360 00000100:00100000:0.0:1776189738.685904:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664399616 00000100:00100000:0.0:1776189738.685921:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664399552 00000100:00100000:0.0:1776189738.685937:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8762 00000100:00100000:0.0:1776189738.685941:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f8f9cfc6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664399360:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189738.685986:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f8f9cfc6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664399360:12345-0@lo:400:kworker/u8:0.0 Request processed in 43us (940us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189738.685991:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8762 00000100:00100000:0.0:1776189738.686002:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8763 00000100:00100000:0.0:1776189738.686005:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000746ef9bb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664399616:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189738.686029:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000746ef9bb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664399616:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (615us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189738.686034:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8763 00000100:00100000:0.0:1776189738.686042:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8764 00000100:00100000:0.0:1776189738.686045:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d1846b23 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664399552:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189738.686064:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d1846b23 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664399552:12345-0@lo:400:kworker/u8:0.0 Request processed in 18us (545us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189738.686069:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8764 00000100:00100000:0.0:1776189738.686088:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d6bc8daa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664399552:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189738.686097:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000050a6a110 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664399616:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189738.686104:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c6bc9cdf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664399360:0@lo:400:kworker/u8:0.0 00000100:00080000:0.0:1776189738.686157:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:2.0:1776189738.686161:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9191 00000100:00080000:0.0:1776189738.686162:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189738.686167:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064636c08 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664399104:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776189738.686197:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189738.686200:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189738.686204:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b7fee202 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664399232:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776189738.686213:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189738.686216:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189738.686218:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d5cfaed8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664399296:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189738.692375:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189738.692440:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005ea3fc0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664399424:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189738.692495:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189738.692524:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005c6b517b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664399680:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189738.692550:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189738.692611:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664399680 00000100:00100000:3.0:1776189738.692661:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7070 00000100:00100000:3.0:1776189738.692666:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000017cf884a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664399680:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776189738.692689:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:3.0:1776189738.692728:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000017cf884a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664399680:12345-0@lo:13:osp-pre-1-1.0 Request processed in 61us (178us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189738.692735:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7070 00000100:00100000:3.0:1776189738.692779:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005c6b517b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664399680:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189738.693252:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189738.693289:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e740dacc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664399488:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189738.693317:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189738.693331:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ca965f6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664399744:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776189738.693353:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189738.694356:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664399424 00000100:00100000:2.0:1776189738.694406:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8765 00000100:00100000:2.0:1776189738.694410:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a829435f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664399424:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189738.694492:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a829435f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664399424:12345-0@lo:400:kworker/u8:0.0 Request processed in 80us (1996us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189738.694498:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8765 00000100:00100000:2.0:1776189738.694522:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664399040 00000100:00100000:2.0:1776189738.694547:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664399488 00000100:00100000:2.0:1776189738.694565:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10230 00000100:00100000:2.0:1776189738.694569:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001cd014bb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664399040:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189738.694601:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001cd014bb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3362:x1862459664399040:12345-0@lo:400:kworker/u8:0.0 Request processed in 31us (1352us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189738.694608:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10230 00000100:00100000:2.0:1776189738.694619:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10231 00000100:00100000:2.0:1776189738.694622:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bd246eb8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664399488:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189738.694647:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bd246eb8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3362:x1862459664399488:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (1330us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189738.694653:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10231 00000100:00100000:0.0:1776189738.694937:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664398976 00000100:00100000:0.0:1776189738.694975:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1593 00000100:00100000:0.0:1776189738.694979:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b1a6ecdc pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664398976:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189738.695012:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b1a6ecdc pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664398976:12345-0@lo:400:kworker/u8:0.0 Request processed in 32us (2644us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189738.695017:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1593 00000100:00100000:3.0:1776189738.695435:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664399744 00000100:00100000:3.0:1776189738.695479:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7071 00000100:00100000:3.0:1776189738.695484:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b1cc0391 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664399744:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776189738.695500:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189738.695533:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b1cc0391 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664399744:12345-0@lo:13:osp-pre-0-1.0 Request processed in 48us (2179us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189738.695539:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7071 00000100:00100000:3.0:1776189738.695576:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c0bb1b99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664398976:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189738.695590:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005ea3fc0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664399424:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189738.696623:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000076bff338 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3362:1862459664399040:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189738.696639:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e740dacc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3362:1862459664399488:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189738.696656:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ca965f6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664399744:0@lo:13:osp-pre-0-1.0 00010000:02020000:2.0:1776189738.758771:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189739.005801:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189739.012399:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189739.268556:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189739.525732:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189739.781629:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189740.036124:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189740.036342:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189740.292519:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189740.548340:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189740.804279:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189741.052539:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189741.060268:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189741.316242:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189741.572296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189741.828345:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189742.076391:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189742.084279:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189742.344292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189742.458560:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189742.458680:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654966912 00000100:00100000:2.0:1776189742.458752:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1594 00000100:00100000:2.0:1776189742.458758:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000090683405 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654966912:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189742.458828:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000090683405 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654966912:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 70us (276us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189742.458836:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1594 00000100:00100000:2.0:1776189742.476510:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189742.476615:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654966976 00000100:00100000:3.0:1776189742.476670:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10232 00000100:00100000:3.0:1776189742.476676:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008be8e55e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654966976:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189742.476761:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008be8e55e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654966976:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 83us (260us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189742.476769:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10232 00000100:00100000:1.0:1776189742.478219:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189742.478845:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654967040 00000100:00100000:2.0:1776189742.478892:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57577 00000100:00100000:2.0:1776189742.478899:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e740dacc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654967040:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776189742.478918:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189742.478985:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e740dacc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654967040:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 84us (770us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189742.478992:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57577 00010000:02020000:2.0:1776189742.596343:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189742.780718:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d97d7d85 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664399872:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189742.780802:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189742.780854:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776189742.780860:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008f3a283b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664399808:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189742.780916:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189742.781516:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664399872 00000100:00100000:1.0:1776189742.781562:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664399808 00000100:00100000:1.0:1776189742.781607:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7072 00000100:00100000:1.0:1776189742.781612:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007ca965f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664399872:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189742.781646:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189742.781707:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007ca965f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664399872:12345-0@lo:13:osp-pre-0-0.0 Request processed in 94us (906us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189742.781713:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7072 00000100:00100000:1.0:1776189742.781730:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7073 00000100:00100000:1.0:1776189742.781734:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005c6b517b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664399808:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189742.781749:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189742.781773:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005c6b517b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664399808:12345-0@lo:13:osp-pre-1-0.0 Request processed in 38us (858us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189742.781779:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7073 00000100:00100000:2.0:1776189742.781919:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008f3a283b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664399808:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189742.781935:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d97d7d85 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664399872:0@lo:13:osp-pre-0-0.0 00010000:02020000:2.0:1776189742.852337:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189743.100615:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189743.108561:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189743.364506:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189743.620700:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:0.0:1776189743.805136:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189743.805144:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189743.805147:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189743.805156:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9885 00000100:00080000:0.0:1776189743.805160:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00080000:0.0:1776189743.805164:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189743.805205:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000081598856 x1862459664400128/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189743.805228:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189743.805231:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189743.805232:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189743.805235:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9885 00000100:00080000:0.0:1776189743.805237:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:0.0:1776189743.805240:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189743.805249:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000060733ebe x1862459664400256/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776189743.805256:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189743.805259:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189743.805260:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189743.805263:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9885 00000100:00080000:0.0:1776189743.805264:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 11s 00000100:00080000:0.0:1776189743.805267:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189743.805276:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000431c2394 x1862459664400320/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776189743.805324:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000081598856 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664400128:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189743.805339:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd150e5b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664399936:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189743.805392:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189743.805419:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000060733ebe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664400256:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189743.805441:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189743.805452:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000431c2394 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664400320:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189743.805496:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189743.805512:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009e8b9a01 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664400384:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189743.805527:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189743.805556:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189743.805572:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000076bff338 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664400640:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189743.805575:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664399936 00000100:00100000:3.0:1776189743.805591:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189743.805604:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1776189743.805609:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0552b59 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664400000:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189743.805610:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7074 00000100:00100000:0.0:1776189743.805614:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d97d7d85 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664399936:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189743.805632:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:0.0:1776189743.805635:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189743.805645:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3e4a548 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664400448:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189743.805662:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189743.805664:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d97d7d85 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664399936:12345-0@lo:13:osp-pre-0-1.0 Request processed in 49us (169us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189743.805671:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7074 00000100:00100000:0.0:1776189743.805691:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664400000 00000100:00100000:3.0:1776189743.805691:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd150e5b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664399936:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189743.805710:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1595 00000100:00100000:0.0:1776189743.805713:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b90a26ba pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664400000:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189743.805727:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664400384 00000100:00100000:0.0:1776189743.805744:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b90a26ba pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3361:x1862459664400000:12345-0@lo:400:kworker/u8:0.0 Request processed in 30us (112us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189743.805750:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1595 00000100:00100000:3.0:1776189743.805766:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664400640 00000100:00100000:0.0:1776189743.805774:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664400128 00000100:00100000:3.0:1776189743.805784:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664400448 00000100:00100000:0.0:1776189743.805786:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664400256 00000100:00100000:0.0:1776189743.805798:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57578 00000100:00100000:3.0:1776189743.805798:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8766 00000100:00100000:0.0:1776189743.805802:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c0bb1b99 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664400128:12345-0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189743.805802:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bf08c9bd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664400384:12345-0@lo:400:kworker/u8:0.0 00010000:02000000:0.0:1776189743.805815:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189743.805830:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bf08c9bd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664400384:12345-0@lo:400:kworker/u8:0.0 Request processed in 27us (274us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189743.805844:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8766 00000100:00100000:0.0:1776189743.805856:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c0bb1b99 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664400128:12345-0@lo:38:kworker/u8:0.0 Request processed in 54us (465us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189743.805857:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8767 00000100:00100000:3.0:1776189743.805860:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000079a89d96 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664400640:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189743.805862:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57578 00000100:00100000:0.0:1776189743.805871:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57579 00000100:00100000:0.0:1776189743.805874:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b7fee202 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664400256:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189743.805881:0:6263:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776189743.805882:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000079a89d96 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664400640:12345-0@lo:400:kworker/u8:0.0 Request processed in 22us (292us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189743.805887:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8767 00000100:00100000:3.0:1776189743.805897:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8768 00000100:00100000:0.0:1776189743.805898:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b7fee202 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3360:x1862459664400256:12345-0@lo:38:kworker/u8:0.0 Request processed in 23us (458us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776189743.805900:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000091ef61c6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664400448:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189743.805903:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57579 00000100:00080000:0.0:1776189743.805931:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:3.0:1776189743.805933:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000091ef61c6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664400448:12345-0@lo:400:kworker/u8:0.0 Request processed in 32us (272us total) trans 0 rc 0/0 00000100:00080000:0.0:1776189743.805934:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189743.805937:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000081598856 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664400128:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189743.805939:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8768 00000100:00080000:0.0:1776189743.805944:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189743.805947:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189743.805949:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000060733ebe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664400256:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776189743.805957:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0552b59 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3361:1862459664400000:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189743.805965:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3e4a548 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664400448:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189743.805973:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009e8b9a01 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664400384:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189743.805980:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000076bff338 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664400640:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189743.807377:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000314a7eb9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664400064:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189743.807471:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189743.807511:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d5cfaed8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664400512:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189743.807533:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189743.807560:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1776189743.807566:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001bdbecc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664400192:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189743.807588:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189743.807605:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c6bc9cdf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664400576:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189743.807623:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189743.807710:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664400320 00000100:00100000:2.0:1776189743.807744:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9192 00000100:00100000:2.0:1776189743.807749:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000064636c08 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664400320:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189743.807765:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189743.807797:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000064636c08 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664400320:12345-0@lo:38:kworker/u8:0.0 Request processed in 48us (2270us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189743.807804:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9192 00000100:00100000:2.0:1776189743.807834:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664400576 00000100:00100000:2.0:1776189743.807855:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8769 00000100:00100000:2.0:1776189743.807859:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009820cd1e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664400576:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189743.807895:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009820cd1e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664400576:12345-0@lo:400:kworker/u8:0.0 Request processed in 35us (271us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189743.807901:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8769 00000100:00100000:2.0:1776189743.807932:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664400064 00000100:00100000:2.0:1776189743.807948:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664400512 00000100:00100000:2.0:1776189743.807957:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664400192 00000100:00100000:2.0:1776189743.807967:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10233 00000100:00100000:1.0:1776189743.808249:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02fb24b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664400704:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776189743.808280:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189743.808310:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664400704 00000100:00100000:1.0:1776189743.808333:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7075 00000100:00100000:1.0:1776189743.808337:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a90403f2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664400704:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776189743.808350:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189743.808393:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a90403f2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664400704:12345-0@lo:13:osp-pre-1-1.0 Request processed in 54us (113us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189743.808400:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7075 00000100:00100000:1.0:1776189743.808437:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02fb24b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664400704:0@lo:13:osp-pre-1-1.0 00000100:00080000:1.0:1776189743.808570:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189743.808574:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189743.808578:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000431c2394 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664400320:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189743.808739:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cd150e5b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664400064:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189743.808773:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cd150e5b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664400064:12345-0@lo:400:kworker/u8:0.0 Request processed in 36us (1303us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189743.819194:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10234 00000100:00100000:3.0:1776189743.819207:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bf60fbf3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664400512:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189743.819298:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bf60fbf3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664400512:12345-0@lo:400:kworker/u8:0.0 Request processed in 95us (11765us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189743.819305:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10234 00000100:00100000:3.0:1776189743.819336:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10235 00000100:00100000:3.0:1776189743.819340:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000067c9d69b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664400192:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189743.819388:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000067c9d69b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3363:x1862459664400192:12345-0@lo:400:kworker/u8:0.0 Request processed in 46us (11799us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189743.819394:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10235 00000100:00100000:3.0:1776189743.819642:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001bdbecc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3363:1862459664400192:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189743.819659:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c6bc9cdf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664400576:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189743.819669:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000314a7eb9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664400064:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189743.819678:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d5cfaed8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664400512:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189743.822507:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10233 00010000:02020000:2.0:1776189743.877729:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189744.124418:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189744.132293:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189744.389001:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189744.644283:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189744.902043:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189745.150408:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189745.156255:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189745.413406:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189745.668300:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189745.924514:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189746.172515:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189746.180288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189746.436252:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189746.694569:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189746.948279:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189747.196637:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189747.204340:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189747.460439:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189747.578499:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189747.578567:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189747.578652:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654967168 00000100:00100000:0.0:1776189747.578720:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10236 00000100:00100000:0.0:1776189747.578727:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001d673174 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654967168:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1776189747.578804:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001d673174 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2278:x1862459654967168:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 76us (311us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189747.578812:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10236 00000100:00100000:2.0:1776189747.578851:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654967104 00000100:00100000:2.0:1776189747.579146:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1596 00000100:00100000:2.0:1776189747.579152:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ef3ea18a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654967104:12345-192.168.203.25@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776189747.579217:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ef3ea18a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2278:x1862459654967104:12345-192.168.203.25@tcp:400:kworker/u8:2.0 Request processed in 64us (650us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189747.579224:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1596 00000100:00100000:1.0:1776189747.579876:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189747.580145:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654967232 00000100:00100000:2.0:1776189747.580201:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57580 00000100:00100000:2.0:1776189747.580206:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c914f1ec pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654967232:12345-192.168.203.25@tcp:38:kworker/u8:2.0 00010000:02000000:2.0:1776189747.580222:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:2.0:1776189747.580272:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c914f1ec pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:2277:x1862459654967232:12345-192.168.203.25@tcp:38:kworker/u8:2.0 Request processed in 64us (398us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189747.580288:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57580 00010000:02020000:2.0:1776189747.716895:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:0.0:1776189747.901024:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776189747.901047:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c1bd7ae7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664400768:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189747.901145:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189747.901222:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a3450613 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664400832:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776189747.901261:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189747.901349:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664400768 00000100:00100000:0.0:1776189747.901406:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664400832 00000100:00100000:2.0:1776189747.901410:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7076 00000100:00100000:2.0:1776189747.901416:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001e60ceac pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664400768:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189747.901422:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7077 00000100:00100000:0.0:1776189747.901426:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000076bff338 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664400832:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776189747.901454:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189747.901505:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000076bff338 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664400832:12345-0@lo:13:osp-pre-1-0.0 Request processed in 77us (243us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189747.901512:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7077 00000100:00100000:0.0:1776189747.901556:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a3450613 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664400832:0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776189747.901869:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189747.902134:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001e60ceac pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664400768:12345-0@lo:13:osp-pre-0-0.0 Request processed in 717us (990us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189747.902142:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7076 00000100:00100000:0.0:1776189747.902876:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c1bd7ae7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664400768:0@lo:13:osp-pre-0-0.0 00010000:02020000:2.0:1776189747.972288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189748.221314:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189748.229909:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189748.484706:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189748.740295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:3.0:1776189748.924491:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02fb24b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664400896:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776189748.924697:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189748.924769:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664400896 00000100:00100000:2.0:1776189748.924818:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7078 00000100:00100000:2.0:1776189748.924823:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ac30dfaa pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664400896:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189748.925148:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189748.925285:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008d6e51a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664400960:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776189748.925315:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189748.925348:0:8238:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664400960 00000100:00100000:0.0:1776189748.925476:0:8238:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7079 00000100:00100000:0.0:1776189748.925480:0:8238:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000314a7eb9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664400960:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776189748.925492:0:8238:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:0.0:1776189748.925522:0:8238:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000314a7eb9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664400960:12345-0@lo:13:osp-pre-1-1.0 Request processed in 41us (207us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189748.925529:0:8238:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7079 00000100:00100000:0.0:1776189748.925572:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008d6e51a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664400960:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189748.929395:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ac30dfaa pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664400896:12345-0@lo:13:osp-pre-0-1.0 Request processed in 4562us (4690us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189748.929410:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7078 00000100:00080000:2.0:1776189748.929558:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189748.929563:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189748.929566:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189748.929572:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9890 00000100:00080000:2.0:1776189748.929575:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:2.0:1776189748.929579:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189748.929595:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002871fa24 x1862459664401152/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189748.929620:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189748.929623:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189748.929624:0:190007:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189748.929627:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9890 00000100:00080000:2.0:1776189748.929629:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00080000:2.0:1776189748.929632:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189748.929641:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006ef3ce93 x1862459664401280/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776189748.929648:0:190007:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776189748.929650:0:190007:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1776189748.929651:0:190007:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776189748.929654:0:190007:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9890 00000100:00080000:2.0:1776189748.929656:0:190007:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 16s 00000100:00080000:2.0:1776189748.929658:0:190007:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189748.929669:0:190007:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008569bb5d x1862459664401344/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189748.929730:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002871fa24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664401152:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189748.929781:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189748.929800:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006ef3ce93 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664401280:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189748.929819:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189748.929834:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008569bb5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664401344:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189748.929854:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189748.929886:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664401152 00000100:00100000:2.0:1776189748.929910:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664401280 00000100:00100000:2.0:1776189748.929925:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57581 00000100:00100000:2.0:1776189748.929944:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000025584b84 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664401152:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189748.929958:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189748.930003:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000025584b84 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664401152:12345-0@lo:38:kworker/u8:1.0 Request processed in 58us (222us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189748.930009:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57581 00000100:00080000:2.0:1776189748.930036:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189748.930039:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189748.930042:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002871fa24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664401152:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189748.930062:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57582 00000100:00100000:2.0:1776189748.930066:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d13f8262 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664401280:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776189748.930072:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189748.930097:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d13f8262 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664401280:12345-0@lo:38:kworker/u8:1.0 Request processed in 30us (278us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189748.930102:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57582 00000100:00080000:2.0:1776189748.930119:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776189748.930122:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:2.0:1776189748.930124:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006ef3ce93 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664401280:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776189748.930316:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eba77318 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664401024:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189748.930346:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189748.930374:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d262f767 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664401472:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189748.930392:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189748.930408:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1776189748.930412:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001b2c3cf4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664401088:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189748.930427:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189748.930437:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f92d37ee pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664401536:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189748.930450:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189748.930474:0:6268:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664401344 00000100:00100000:0.0:1776189748.930496:0:6268:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9193 00000100:00100000:0.0:1776189748.930500:0:6268:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000092244372 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664401344:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776189748.930508:0:6268:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189748.930537:0:6268:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000092244372 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:0+-99:3360:x1862459664401344:12345-0@lo:38:kworker/u8:1.0 Request processed in 37us (683us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189748.930544:0:6268:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9193 00000100:00100000:0.0:1776189748.930569:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664401472 00000100:00100000:0.0:1776189748.930590:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8770 00000100:00100000:0.0:1776189748.930594:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001bdbecc8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664401472:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189748.930625:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001bdbecc8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664401472:12345-0@lo:400:kworker/u8:1.0 Request processed in 31us (233us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189748.930630:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8770 00000100:00100000:0.0:1776189748.930650:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664401088 00000100:00100000:0.0:1776189748.930665:0:6263:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664401536 00000100:00100000:0.0:1776189748.930674:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10237 00000100:00100000:0.0:1776189748.930676:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000431c2394 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664401088:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189748.930695:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000431c2394 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664401088:12345-0@lo:400:kworker/u8:1.0 Request processed in 18us (269us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189748.930698:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10237 00000100:00100000:0.0:1776189748.930704:0:6263:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10238 00000100:00100000:0.0:1776189748.930706:0:6263:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009e8b9a01 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664401536:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189748.930717:0:6263:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009e8b9a01 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664401536:12345-0@lo:400:kworker/u8:1.0 Request processed in 10us (267us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189748.930722:0:6263:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10238 00000100:00100000:0.0:1776189748.930735:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000066c7cbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664401408:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189748.930749:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189748.930758:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776189748.930761:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bc860b89 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664401216:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189748.930813:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189748.930827:0:44530:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664401408 00000100:00100000:0.0:1776189748.930839:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8771 00000100:00100000:0.0:1776189748.930841:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b3e4a548 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664401408:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189748.930858:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b3e4a548 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664401408:12345-0@lo:400:kworker/u8:1.0 Request processed in 16us (109us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189748.930861:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8771 00000100:00100000:0.0:1776189748.930871:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664401216 00000100:00100000:0.0:1776189748.930882:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10239 00000100:00100000:0.0:1776189748.930885:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b0552b59 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664401216:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189748.930908:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b0552b59 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664401216:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (94us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189748.930927:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10239 00000100:00100000:0.0:1776189748.930957:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bc860b89 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664401216:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189748.930966:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000066c7cbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664401408:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189748.930989:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001b2c3cf4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664401088:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189748.930996:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f92d37ee pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664401536:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776189748.931003:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d262f767 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664401472:0@lo:400:kworker/u8:1.0 00000100:00080000:0.0:1776189748.931028:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189748.931031:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189748.931034:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008569bb5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664401344:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776189748.933455:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664401024 00000100:00100000:2.0:1776189748.933518:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1597 00000100:00100000:2.0:1776189748.933524:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005c923f64 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664401024:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776189748.933604:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005c923f64 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664401024:12345-0@lo:400:kworker/u8:1.0 Request processed in 80us (3258us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189748.933611:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1597 00000100:00100000:0.0:1776189748.933941:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eba77318 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664401024:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776189748.935727:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02fb24b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664400896:0@lo:13:osp-pre-0-1.0 00010000:02020000:2.0:1776189748.996342:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189749.247144:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189749.253138:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189749.508292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189749.764288:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189750.020837:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189750.271156:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189750.278233:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189750.532301:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189750.788430:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189751.044529:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189751.292759:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:2.0:1776189751.300646:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189751.556296:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189751.812922:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:2.0:1776189752.068292:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189752.317670:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189752.325215:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189752.580457:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189752.700285:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189752.700337:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1776189752.700439:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654967360 00000100:00100000:2.0:1776189752.700500:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10240 00000100:00100000:2.0:1776189752.700507:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003d88b4c1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654967360:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776189752.700637:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003d88b4c1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2281:x1862459654967360:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 129us (299us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189752.700654:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10240 00000100:00100000:0.0:1776189752.701344:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654967296 00000100:00100000:0.0:1776189752.701390:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1598 00000100:00100000:0.0:1776189752.701395:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cd00b680 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654967296:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189752.701444:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cd00b680 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2280:x1862459654967296:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 48us (1169us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189752.701451:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1598 00000100:00100000:1.0:1776189752.704643:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1776189752.706767:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654967424 00000100:00100000:0.0:1776189752.706837:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57583 00000100:00100000:0.0:1776189752.706844:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000132f6f38 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654967424:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776189752.706863:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:0.0:1776189752.706937:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000132f6f38 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2277:x1862459654967424:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 93us (2301us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189752.706945:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57583 00010000:02020000:0.0:1776189752.836701:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189753.027341:0:3362:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776189753.027365:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ae8c4d3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664401600:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189753.027435:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189753.027543:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664401600 00000100:00100000:3.0:1776189753.027593:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7080 00000100:00100000:3.0:1776189753.027599:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006ef3ce93 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664401600:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776189753.027630:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:3.0:1776189753.027693:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006ef3ce93 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664401600:12345-0@lo:13:osp-pre-0-0.0 Request processed in 93us (258us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189753.027700:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7080 00000100:00100000:3.0:1776189753.027745:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ae8c4d3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664401600:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189753.030361:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776189753.030388:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009626151d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664401664:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189753.030457:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189753.032192:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664401664 00000100:00100000:1.0:1776189753.032261:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7081 00000100:00100000:1.0:1776189753.032268:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002871fa24 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664401664:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189753.032299:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189753.032378:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002871fa24 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664401664:12345-0@lo:13:osp-pre-1-0.0 Request processed in 108us (1920us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189753.032385:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7081 00000100:00100000:2.0:1776189753.033951:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009626151d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664401664:0@lo:13:osp-pre-1-0.0 00010000:02020000:0.0:1776189753.093561:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189753.343789:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189753.353412:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189753.604369:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189753.862334:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:0.0:1776189754.044586:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189754.044593:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189754.044597:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189754.044604:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9895 00000100:00080000:0.0:1776189754.044609:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1776189754.044624:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:2.0:1776189754.044636:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000060733ebe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664401728:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189754.044705:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189754.044719:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000002e1c8a0 x1862459664401856/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189754.044741:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00080000:0.0:1776189754.044749:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1776189754.044750:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000081598856 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664401792:0@lo:400:kworker/u8:0.0 00000100:00080000:0.0:1776189754.044756:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189754.044758:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189754.044763:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9895 00000100:00080000:0.0:1776189754.044765:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1776189754.044770:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189754.044774:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000002e1c8a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664401856:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189754.044784:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000034dff7c x1862459664402048/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189754.044786:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776189754.044796:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776189754.044799:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1776189754.044800:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776189754.044804:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9895 00000100:00080000:0.0:1776189754.044807:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1776189754.044809:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776189754.044810:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:0.0:1776189754.044822:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001a95cedd x1862459664402112/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776189754.044832:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664401728 00000100:00100000:1.0:1776189754.044837:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000034dff7c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664402048:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189754.044864:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189754.044884:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1599 00000100:00100000:1.0:1776189754.044887:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001a95cedd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664402112:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189754.044893:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f6dc323d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664401728:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189754.044903:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664401792 00000100:00100000:1.0:1776189754.044914:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189754.044929:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664401856 00000100:00100000:2.0:1776189754.044958:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f6dc323d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664401728:12345-0@lo:400:kworker/u8:0.0 Request processed in 65us (253us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189754.044965:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1599 00000100:00100000:2.0:1776189754.044999:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d818a4c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664401920:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189754.045025:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189754.045037:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003719584e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664402240:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189754.045057:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001abe0d42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664402304:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189754.045090:0:6264:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664402048 00000100:00100000:3.0:1776189754.045302:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189754.045710:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189754.045734:0:3364:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1776189754.045743:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000064376a3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664402176:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189754.045773:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189754.045793:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d0e96547 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664402432:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189754.045813:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189754.045855:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088c86c02 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664401984:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189754.055796:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664402240 00000100:00100000:2.0:1776189754.055858:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664402176 00000100:00100000:2.0:1776189754.055875:0:8234:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664402432 00000100:00100000:2.0:1776189754.055900:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8772 00000100:00100000:2.0:1776189754.055906:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001ae8c4d3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664402240:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189754.055980:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001ae8c4d3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664402240:12345-0@lo:400:kworker/u8:0.0 Request processed in 74us (10678us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189754.055988:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8772 00000100:00100000:2.0:1776189754.056003:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8773 00000100:00100000:2.0:1776189754.056006:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004abaab4f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664402176:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189754.056046:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004abaab4f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3364:x1862459664402176:12345-0@lo:400:kworker/u8:0.0 Request processed in 38us (10273us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189754.056051:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8773 00000100:00100000:2.0:1776189754.056061:0:8234:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8774 00000100:00100000:2.0:1776189754.056065:0:8234:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fe93ff8e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664402432:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189754.056090:0:8234:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fe93ff8e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664402432:12345-0@lo:400:kworker/u8:0.0 Request processed in 25us (10277us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189754.056096:0:8234:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8774 00000100:00100000:2.0:1776189754.056136:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189754.059360:0:190070:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664402304 00000100:00100000:1.0:1776189754.059423:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10241 00000100:00100000:1.0:1776189754.059429:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ff836739 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664401792:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189754.059502:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10242 00000100:00100000:0.0:1776189754.059507:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007c461eb1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664402304:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189754.059515:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ff836739 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664401792:12345-0@lo:400:kworker/u8:0.0 Request processed in 86us (14734us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189754.059522:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10241 00000100:00100000:0.0:1776189754.059540:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007c461eb1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3361:x1862459664402304:12345-0@lo:400:kworker/u8:0.0 Request processed in 33us (13832us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189754.059540:0:190070:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57584 00000100:00100000:1.0:1776189754.059544:0:190070:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002cc16798 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664401856:12345-0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776189754.059546:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10242 00000100:00100000:0.0:1776189754.059558:0:6264:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57585 00010000:02000000:1.0:1776189754.059559:0:190070:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189754.059561:0:6264:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ca2555cf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664402048:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776189754.059568:0:6264:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189754.059598:0:190070:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002cc16798 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3360:x1862459664401856:12345-0@lo:38:kworker/u8:0.0 Request processed in 53us (14790us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189754.059604:0:190070:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57584 00000100:00100000:2.0:1776189754.059708:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a20c1a6c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664402368:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189754.059746:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189754.059780:0:44532:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664402368 00000100:00100000:2.0:1776189754.059806:0:44532:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8775 00000100:00100000:2.0:1776189754.059810:0:44532:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006bbdf9dc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664402368:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189754.059867:0:44532:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006bbdf9dc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+5:3362:x1862459664402368:12345-0@lo:400:kworker/u8:0.0 Request processed in 56us (120us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189754.059873:0:44532:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8775 00000100:00100000:2.0:1776189754.059913:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a20c1a6c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3362:1862459664402368:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189754.059942:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000081598856 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664401792:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189754.059951:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000060733ebe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664401728:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189754.059981:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000064376a3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664402176:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189754.059992:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d0e96547 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664402432:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189754.060001:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003719584e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3364:1862459664402240:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189754.060036:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001abe0d42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3361:1862459664402304:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189754.060170:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664401920 00000100:00100000:2.0:1776189754.060216:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7082 00000100:00100000:2.0:1776189754.060220:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a41f61f7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664401920:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189754.060235:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189754.060269:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a41f61f7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664401920:12345-0@lo:13:osp-pre-1-1.0 Request processed in 48us (15245us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189754.060275:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7082 00000100:00100000:2.0:1776189754.060961:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776189754.060974:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d669bbbc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664402496:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189754.061006:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189754.061046:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664402496 00000100:00100000:2.0:1776189754.061071:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7083 00000100:00100000:2.0:1776189754.061075:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000089e2759f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664402496:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776189754.061086:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:2.0:1776189754.061116:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000089e2759f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664402496:12345-0@lo:13:osp-pre-0-1.0 Request processed in 40us (110us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189754.061276:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7083 00000100:00100000:2.0:1776189754.061315:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d669bbbc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664402496:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776189754.061360:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d818a4c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664401920:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776189754.061403:0:9639:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664401984 00000100:00100000:2.0:1776189754.061421:0:9639:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10243 00000100:00100000:2.0:1776189754.061424:0:9639:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000633179d8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664401984:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189754.061460:0:9639:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000633179d8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3362:x1862459664401984:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (5324us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189754.061466:0:9639:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10243 00000100:00100000:2.0:1776189754.061493:0:3362:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088c86c02 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3362:1862459664401984:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189754.061536:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664402112 00000100:00100000:2.0:1776189754.061556:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9194 00000100:00100000:2.0:1776189754.061560:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000627f6a9d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664402112:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189754.061571:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776189754.067613:0:6264:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ca2555cf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3360:x1862459664402048:12345-0@lo:38:kworker/u8:0.0 Request processed in 8041us (22739us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776189754.067719:0:6264:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57585 00000100:00100000:2.0:1776189754.069257:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000627f6a9d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664402112:12345-0@lo:38:kworker/u8:0.0 Request processed in 7689us (24339us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189754.069269:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9194 00000100:00080000:1.0:1776189754.072694:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189754.072705:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189754.072712:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000034dff7c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664402048:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776189754.072737:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189754.072740:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189754.072743:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000002e1c8a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664401856:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776189754.072780:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189754.072783:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189754.072786:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001a95cedd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664402112:0@lo:38:kworker/u8:0.0 00010000:02020000:0.0:1776189754.116330:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189754.365070:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189754.374050:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189754.630308:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189754.884349:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189755.141205:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189755.390103:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189755.397143:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189755.652342:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189755.908404:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189756.164295:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189756.412918:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189756.420297:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189756.676488:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189756.932410:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189757.190322:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189757.436994:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189757.444555:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189757.700309:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:1.0:1776189757.819136:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189757.819224:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1776189757.819335:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654967552 00000100:00100000:3.0:1776189757.819407:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10244 00000100:00100000:3.0:1776189757.819413:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e87a98c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654967552:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776189757.819500:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e87a98c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:ab98a7bc-55da-40f7-b8ba-dc1b8e6ca19f+13:2279:x1862459654967552:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 86us (277us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189757.819508:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10244 00000100:00100000:0.0:1776189757.819619:0:186085:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654967488 00000100:00100000:0.0:1776189757.819664:0:186085:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1600 00000100:00100000:0.0:1776189757.819669:0:186085:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000078287bc4 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654967488:12345-192.168.203.25@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776189757.819707:0:186085:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000078287bc4 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:657b76c8-4dea-4c1c-9b97-f2ea2534b8c2+9:2279:x1862459654967488:12345-192.168.203.25@tcp:400:kworker/u8:3.0 Request processed in 37us (578us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189757.819715:0:186085:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1600 00000100:00100000:3.0:1776189757.824270:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1776189757.825412:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459654967616 00000100:00100000:1.0:1776189757.825470:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 57586 00000100:00100000:1.0:1776189757.825478:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dde983b4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654967616:12345-192.168.203.25@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776189757.825497:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 192.168.203.25@tcp (stopping) 00000100:00100000:1.0:1776189757.825570:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dde983b4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:2277:x1862459654967616:12345-192.168.203.25@tcp:38:kworker/u8:3.0 Request processed in 92us (1307us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189757.825578:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 57586 00010000:02020000:0.0:1776189757.958183:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189758.140405:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d781d9c8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664402560:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776189758.140475:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776189758.140521:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776189758.140527:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000043af2eb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664402624:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189758.140557:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189758.140636:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664402560 00000100:00100000:1.0:1776189758.140684:0:40419:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664402624 00000100:00100000:1.0:1776189758.140702:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7084 00000100:00100000:1.0:1776189758.140708:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d669bbbc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664402560:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776189758.140735:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189758.140778:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d669bbbc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664402560:12345-0@lo:13:osp-pre-0-0.0 Request processed in 69us (304us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189758.140785:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7084 00000100:00100000:1.0:1776189758.140800:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7085 00000100:00100000:1.0:1776189758.140804:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008fea0698 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664402624:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189758.140819:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:1.0:1776189758.140848:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008fea0698 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664402624:12345-0@lo:13:osp-pre-1-0.0 Request processed in 43us (291us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189758.140853:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7085 00000100:00100000:1.0:1776189758.140894:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000043af2eb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664402624:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776189758.140906:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d781d9c8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664402560:0@lo:13:osp-pre-0-0.0 00010000:02020000:0.0:1776189758.212306:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189758.462470:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189758.468919:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189758.724425:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189758.981620:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00080000:1.0:1776189759.164417:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0001_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189759.164424:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189759.164427:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189759.164434:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0001: connect to NID 0@lo last attempt 9901 00000100:00080000:1.0:1776189759.164437:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189759.164443:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0001: import 00000000764e790c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189759.164463:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d818a4c0 x1862459664402816/t0(0) o38->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189759.164496:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-lwp-OST0000_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189759.164499:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189759.164500:0:134480:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189759.164504:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-lwp-OST0000: connect to NID 0@lo last attempt 9901 00000100:00080000:1.0:1776189759.164505:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189759.164509:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-lwp-OST0000: import 00000000f0a70bb7 using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189759.164519:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008d6e51a5 x1862459664402944/t0(0) o38->lustre-MDT0001-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776189759.164526:0:134480:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776189759.164529:0:134480:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1776189759.164530:0:134480:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776189759.164533:0:134480:0:(import.c:534:import_select_connection()) lustre-MDT0001-osp-MDT0000: connect to NID 0@lo last attempt 9901 00000100:00080000:1.0:1776189759.164535:0:134480:0:(import.c:581:import_select_connection()) lustre-MDT0001-osp-MDT0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776189759.164538:0:134480:0:(import.c:615:import_select_connection()) lustre-MDT0001-osp-MDT0000: import 000000006e48578c using connection 192.168.203.125@tcp/0@lo 00000100:00100000:1.0:1776189759.164547:0:134480:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a3450613 x1862459664403008/t0(0) o38->lustre-MDT0001-osp-MDT0000@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776189759.164629:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009626151d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664402880:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189759.164705:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189759.164733:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d6bc8daa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664403264:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189759.164769:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189759.164789:0:3361:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1776189759.164795:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008f3a283b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664403072:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189759.164816:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189759.164828:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d34d173 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664403328:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189759.164845:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189759.164874:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664402880 00000100:00100000:1.0:1776189759.164923:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10245 00000100:00100000:1.0:1776189759.164929:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000001252b58 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664402880:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189759.164976:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000001252b58 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3361:x1862459664402880:12345-0@lo:400:kworker/u8:0.0 Request processed in 47us (272us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189759.164983:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10245 00000100:00100000:1.0:1776189759.165039:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009626151d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3361:1862459664402880:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189759.165063:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d818a4c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664402816:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189759.165086:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189759.165107:0:45899:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664402816 00000100:00100000:1.0:1776189759.165126:0:45899:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57587 00000100:00100000:1.0:1776189759.165130:0:45899:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000007875e77 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664402816:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189759.165146:0:45899:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189759.165195:0:45899:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000007875e77 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3360:x1862459664402816:12345-0@lo:38:kworker/u8:0.0 Request processed in 42us (87us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189759.165201:0:45899:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57587 00000100:00100000:1.0:1776189759.165218:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008d6e51a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664402944:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189759.165240:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189759.165255:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664402944 00000100:00100000:1.0:1776189759.165270:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57588 00000100:00100000:1.0:1776189759.165273:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cc3ee522 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664402944:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776189759.165280:0:6262:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776189759.165301:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cc3ee522 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3360:x1862459664402944:12345-0@lo:38:kworker/u8:0.0 Request processed in 28us (62us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776189759.165307:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57588 00000100:00100000:1.0:1776189759.165322:0:3360:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a3450613 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664403008:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776189759.165345:0:3360:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776189759.165368:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000764e790c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189759.165370:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189759.165374:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d818a4c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0001_UUID:3360:1862459664402816:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776189759.165381:0:3360:0:(import.c:86:import_set_state_nolock()) 00000000f0a70bb7 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776189759.165383:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:1.0:1776189759.165385:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008d6e51a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-lwp-OST0000_UUID:3360:1862459664402944:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189759.165404:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664403264 00000100:00100000:2.0:1776189759.165428:0:44533:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664403072 00000100:00100000:1.0:1776189759.165436:0:44534:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664403328 00000100:00100000:2.0:1776189759.165442:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8776 00000100:00100000:2.0:1776189759.165446:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000c5611bf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664403264:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189759.165454:0:44534:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8777 00000100:00100000:1.0:1776189759.165458:0:44534:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cae4ee67 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664403072:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776189759.165481:0:44534:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cae4ee67 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664403072:12345-0@lo:400:kworker/u8:0.0 Request processed in 22us (665us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189759.165481:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000c5611bf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664403264:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (712us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189759.165487:0:44534:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8777 00000100:00100000:3.0:1776189759.165530:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000043af2eb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664402688:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189759.165561:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189759.165575:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005ea3fc0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664403136:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189759.165594:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189759.165606:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f761e567 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664403392:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776189759.165624:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189759.165638:0:3363:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1776189759.165644:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f0cb0abd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664402752:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189759.165663:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189759.165676:0:3363:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000050a6a110 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664403200:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189759.165696:0:3363:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776189759.165718:0:186088:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664402688 00000100:00100000:3.0:1776189759.165744:0:186088:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1601 00000100:00100000:3.0:1776189759.165746:0:186088:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003afac4ae pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664402688:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776189759.165777:0:186088:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003afac4ae pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521+18:3363:x1862459664402688:12345-0@lo:400:kworker/u8:0.0 Request processed in 29us (215us total) trans 0 rc 0/0 00000100:00100000:3.0:1776189759.165782:0:186088:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1601 00000100:00100000:3.0:1776189759.165808:0:186995:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664403136 00000100:00100000:2.0:1776189759.171071:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8776 00000100:00100000:2.0:1776189759.171119:0:44533:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8778 00000100:00100000:2.0:1776189759.171128:0:44533:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007633d4ee pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664403328:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189759.171234:0:44533:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007633d4ee pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3361:x1862459664403328:12345-0@lo:400:kworker/u8:0.0 Request processed in 107us (6388us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189759.171241:0:44533:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8778 00000100:00100000:2.0:1776189759.171277:0:6267:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664403008 00000100:00100000:2.0:1776189759.171326:0:6267:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9195 00000100:00100000:2.0:1776189759.171330:0:6267:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000039973ea1 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664403008:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776189759.171346:0:6267:0:(ldlm_lib.c:1140:target_handle_connect()) lustre-MDT0001: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776189759.171383:0:6267:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000039973ea1 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:0+-99:3360:x1862459664403008:12345-0@lo:38:kworker/u8:0.0 Request processed in 52us (6038us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776189759.171390:0:6267:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9195 00000100:00100000:2.0:1776189759.171630:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008f3a283b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664403072:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189759.173472:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d34d173 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664403328:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189759.173485:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d6bc8daa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3361:1862459664403264:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189759.173565:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664403392 00000100:00100000:2.0:1776189759.173608:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7086 00000100:00100000:2.0:1776189759.173614:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000046646cdd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664403392:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776189759.174909:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189759.174986:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000046646cdd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3363:x1862459664403392:12345-0@lo:13:osp-pre-1-1.0 Request processed in 1370us (9360us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189759.174995:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7086 00000100:00100000:2.0:1776189759.175053:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664402752 00000100:00100000:2.0:1776189759.175089:0:6262:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664403200 00000100:00100000:2.0:1776189759.175108:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10246 00000100:00100000:2.0:1776189759.175112:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006392c95c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664402752:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189759.175149:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006392c95c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3363:x1862459664402752:12345-0@lo:400:kworker/u8:0.0 Request processed in 36us (9485us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189759.175155:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10246 00000100:00100000:2.0:1776189759.175167:0:6262:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10247 00000100:00100000:2.0:1776189759.175170:0:6262:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007f9bf6ea pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664403200:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189759.175217:0:6262:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007f9bf6ea pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3363:x1862459664403200:12345-0@lo:400:kworker/u8:0.0 Request processed in 45us (9520us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189759.175222:0:6262:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10247 00000100:00100000:2.0:1776189759.175284:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f0cb0abd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3363:1862459664402752:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189759.175295:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000050a6a110 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3363:1862459664403200:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189759.175846:0:44530:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8779 00000100:00100000:0.0:1776189759.175854:0:44530:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e2a0d74f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664403136:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776189759.175898:0:44530:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e2a0d74f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3363:x1862459664403136:12345-0@lo:400:kworker/u8:0.0 Request processed in 45us (10304us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189759.175904:0:44530:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8779 00000100:00100000:0.0:1776189759.175945:0:3364:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000036ddffa8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664403456:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776189759.175984:0:3364:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776189759.176024:0:8239:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664403456 00000100:00100000:0.0:1776189759.176070:0:8239:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7087 00000100:00100000:0.0:1776189759.176075:0:8239:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000492f27d6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664403456:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776189759.176090:0:8239:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:0.0:1776189759.176125:0:8239:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000492f27d6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3364:x1862459664403456:12345-0@lo:13:osp-pre-0-1.0 Request processed in 49us (142us total) trans 0 rc 0/0 00000100:00100000:0.0:1776189759.176132:0:8239:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7087 00000100:00100000:0.0:1776189759.176167:0:3364:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000036ddffa8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3364:1862459664403456:0@lo:13:osp-pre-0-1.0 00000100:00080000:0.0:1776189759.176249:0:3360:0:(import.c:86:import_set_state_nolock()) 000000006e48578c lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776189759.176254:0:3360:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.203.125@tcp failed (-19) 00000100:00100000:0.0:1776189759.176258:0:3360:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a3450613 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3360:1862459664403008:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776189759.177812:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000043af2eb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4cf8dc14-bb17-41bb-9ca1-9ce2a255b521:3363:1862459664402688:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189759.177845:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005ea3fc0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3363:1862459664403136:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776189759.177865:0:3363:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f761e567 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3363:1862459664403392:0@lo:13:osp-pre-1-1.0 00010000:02020000:0.0:1776189759.236357:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189759.484461:0:6251:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189759.492291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189759.749245:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189760.006308:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189760.260430:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189760.508474:0:19746:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189760.516271:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189760.772306:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189761.028291:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189761.284345:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189761.532399:0:7268:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189761.540326:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189761.796307:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189762.054063:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189762.308247:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:00010000:2.0:1776189762.556525:0:6250:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:02020000:0.0:1776189762.564372:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189762.820268:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00010000:02020000:0.0:1776189763.076482:0:189743:0:(ldlm_resource.c:1191:__ldlm_namespace_free()) 0-0: Forced cleanup waiting for lustre-MDT0000-osp-MDT0001 namespace with 2 resources in use, (rc=-110) 00000100:00100000:2.0:1776189763.260477:0:3361:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012a3bc9a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664403520:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189763.260556:0:3361:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189763.260807:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664403520 00000100:00100000:0.0:1776189763.260817:0:3362:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c7e233 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3362:1862459664403584:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776189763.260852:0:3362:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776189763.262593:0:15862:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862459664403584 00000100:00100000:1.0:1776189763.262658:0:15862:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7088 00000100:00100000:1.0:1776189763.262664:0:15862:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007d3c1492 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664403520:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776189763.262695:0:15862:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 142272 pending 0 free 3923664896 avail 3692142592 00000100:00100000:2.0:1776189763.262732:0:40419:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7089 00000100:00100000:2.0:1776189763.262738:0:40419:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007dae5657 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664403584:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776189763.262758:0:40419:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 0 granted 240576 pending 0 free 3923623936 avail 3692101632 00000100:00100000:1.0:1776189763.267658:0:15862:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007d3c1492 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3361:x1862459664403520:12345-0@lo:13:osp-pre-1-0.0 Request processed in 4985us (7094us total) trans 0 rc 0/0 00000100:00100000:1.0:1776189763.267674:0:15862:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7088 00000100:00100000:0.0:1776189763.267731:0:3361:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012a3bc9a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3361:1862459664403520:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776189763.268839:0:40419:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007dae5657 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3362:x1862459664403584:12345-0@lo:13:osp-pre-0-0.0 Request processed in 6096us (7984us total) trans 0 rc 0/0 00000100:00100000:2.0:1776189763.268849:0:40419:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7089 Debug log: 35993 lines, 35993 kept, 0 dropped, 0 bad.